Keywords

1 Introduction

Surveys of industry representatives [1, 2] as well as reports by international organizations [3, 4] and even EU legislation [5, 6] have highlighted a demand for employees with critical thinking (CT) skills in the workforce. However, despite this demand, several studies conducted by non-governmental organizations [7, 8] have identified CT as one of the biggest current and future gaps in employees’ abilities. The data from these studies suggest that organizations need to focus on developing soft skills like critical thinking [7].

The gap between the demand for CT skills and the number of employees who possess them is a major concern for industry [1, 2, 7, 8]. To address this issue, there is a call for developing effective methods for engaging employees in CT skills learning at the workplace. The development of CT skills is a complex and cognitively demanding learning activity, especially in the context of workplace learning [9].

Games-based learning has been proposed as an effective solution for workplace learning of CT skills, as games provide both motivation and engagement in an active learning environment and have a positive impact on learning goals [10]. Games are often perceived as enjoyable learning tools and contribute to the improvement of critical thinking and reasoning [10]. Previous research has also shown that games can have a beneficial impact on cognitive outcomes, including CT [9, 11].

Based on the demand from industry, the identified lack of CT skills among employees, and the specific aspects of workplace learning, a game for CT skills development training was designed, drawing on instructional interventions identified by Abrami et al. [12]. These interventions include opportunities for dialogue (e.g., discussion), authentic or situated problems and examples, and mentorship, which are integrated into the game design.

To enhance the game’s usability, a gamification platform called Bet-N-Learn (betnlearn.com) was used for constructing the game. Bet-N-Learn is a web-based software that allows for the creation of a game with multiple rounds, each containing a multiple-choice question (MCQ) followed by answer options. Players, who are authorized to access the game via their smartphones or other devices with internet access, can place bets on answer options using game currency (coins). The game facilitator has control over starting and terminating the game, and the platform collects and displays real-time data on players’ results, including bets on answer options, time taken to submit bets, and more, allowing for evaluation of the level of certainty of players.

To gain insights into players’ engagement while playing the game, a qualitative study was conducted to explore players’ experiences. The research question guiding this study was: How do game players describe their engagement when playing the game?

This paper is organized into three sections: related studies (Sect. 2), where examples of research on games used for CT development are provided; the methodology (Sect. 3), where the quality methodology used in the study is explained; and finally, the results and conclusions are presented and discussed (Sect. 4).

2 Related Research

In their systematic literature review, Vlachopoulos and Makri [10] found that engagement is considered a central factor in the majority of game-based learning studies they reviewed. However, research papers focusing on games that develop CT skills often lack qualitative studies on engagement in CT games. For example, Halpern et al. [9] evaluated engagement using quantitative methods, comparing data from tutoring conditions with varying levels of active engagement [9]. Similarly, Stanley and Latimer [13] used questionnaires to assess game enjoyment, which can suggest a correlation between enjoyment and engagement, but may not fully capture the level of engagement during gameplay. Therefore, this study aims to contribute to fulfilling this research gap by examining the qualitative study of engagement in CT games.

3 Methodology

3.1 The Game

The game was designed to be interactive and challenging, with multiple rounds of MCQs that required players to apply CT skills. The game tasks were designed in accordance with the three sections of the Watson-Glaser Critical Thinking Appraisal (WGCTA) [14], specifically evaluation of arguments (EA), recognition of assumptions (RA), and drawing conclusions (DC), while following the guidelines outlined by Morrison and Free [15] and ensuring that the tasks were authentic to the players [12]. WGCTA was chosen because it demonstrates widespread international utilization and enduring longevity, with its first standardized version dating back to 1951 [16]. Now, it has emerged as one of the most commonly employed assessments in business [17].

The game consisted of twelve rounds, each presented as a question (EA), proposition (RA), or statement (DC), with four multiple-choice answers provided for each round. The rounds were grouped into three sets: the first set (S1) and the third set (S3) each contained three MCQs (one EA, one RA, and one DC), while the second set (S2) contained six MCQs (two EA, two RA, and two DC).

The following example is one of the EA MCQ. Question: “Should all the managers in our company have a higher education?” Four options of answer on the choice of the player: (1) “Yes, because we have the financial resources to hire more highly educated workers.” (2) “No, because managers with higher education are a lot more expensive.” (3) “Yes, because highly qualified personnel ensure a high level of productivity in organizations.” (4) “No, because excessive qualification can compromise motivation and so reduce efficiency.”

In the present study, the allocation of coins for wagering on answer options was determined by means of a formula. Specifically, the number of available coins was calculated as the difference between the total number of answer options and one. For example, when four answer options were provided, players had three coins at their disposal. This restriction was implemented with the intention of promoting thoughtful consideration of choices, rather than hasty and indiscriminate wagers on all possible answers.

Russian was chosen for tasks and instruction preparation due to the participants’ proficiency or native speaker status. According to Manalo and Sheppard [18], the language of instruction can impact the ability of individuals to fully demonstrate their critical thinking skills, with poorer performance likely when instructions are given in a foreign language compared to their native language.

3.2 The Training

The CT skills development training using the game was conducted as a standalone intervention for employees of the company. The training comprised five sessions held over a period of five consecutive days, each session spanning three hours. The sessions were designed to explain the relevant concepts of critical thinking (such as deductive reasoning) by providing appropriate examples and addressing the participants’ queries.

During three of the five sessions, the participants engaged in gameplay. They were randomly divided into four groups of three players each and instructed to assume the roles of a company’s board of directors. The groups were then asked to engage in discussions about various important issues that the company faced and make decisions accordingly.

Each group of players engaged in discussions of MCQs in order to identify the correct answer(s). Each player within a group had individual access to the platform and was able to place bets on any of the answer options, regardless of other players’ choices.

After each set of rounds, all training participants convened to discuss the MCQs they had encountered during the game and to evaluate their performance.

Over the course of five days, participants were given three sets of rounds that were administered over three days, with S1 and S3 on the first and third day respectively, and S2 on the second day.

3.3 Design and Participants

The study included 12 adult members of the Management Board of a pet food production company with international headquarters located in Ukraine, representing diverse backgrounds. The training was conducted over the course of five days in April 2021. Prior to administering the survey, verbal consent was obtained from the participants for anonymous usage of the survey data in our research. Written consents were received by mail in May 2021.

3.4 Data Collection

The data collection for this study involved an online survey administered through Google Forms. Participants were instructed to use Incognito mode to ensure anonymity. The survey was administered after the completion of the final training session. Out of the 12 participants, nine feedback forms were received, yielding a response rate of 75%. The survey included various forms of anonymous feedback, such as free-form responses and open-ended questions. Participants were also asked to evaluate how easy it was to interact with the platform interface and how intuitive it was to use, using a Likert-type scale ranging from 1 (Very Difficult) to 5 (Very Easy) and 1 (Not intuitive) to 5 (Intuitive), respectively.

The open-ended questions asked participants to describe how this training differed from others they had attended and what they found to be the most valuable part of the training. All survey questions were presented in Russian, and responses were received in Russian. Later, the responses were translated into English for analysis. We utilized an inductive data analysis approach [19] with the assistance of Atlas.ti software.

4 Results and Conclusions

The participants reported finding the game engaging (e.g., “we got carried away, I was very involved”; “it was interesting and unusual”; “Interactive - dialogues and feedback”). They also reported finding the game interesting (e.g., “Interesting gamification”; “it was very accessible, informative, interesting”).

The participants reported valuable learning outcomes (e.g., "Learn to distinguish where manipulation can be and in what forms"; "an understanding of how ambiguity can distort the real picture and mislead or draw the wrong conclusion"; "Very informative and interesting! Absolutely ‘adjective’ to my work, life situations"). They reported learning about CT, problem-solving, and identifying logical fallacies.

As a part of the survey, all participants were asked if they had experience with similar gaming platforms in other trainings, and none of them reported any prior experience. Moreover, none of the participants had previous experience with the specific gamification platform used in this training.

In order to assess the usability of the gamification platform, participants were asked to evaluate the ease of interaction with the platform’s interface and its intuitiveness. The Likert scale was used, ranging from 1 (Very Difficult/Not intuitive) to 5 (Very Easy/Intuitive). The results showed that the majority of participants found the platform easy to interact with, with 56% rating it as “Very Easy” (score of 5) and 44% rating it as “Easy” (score of 4). Similarly, the majority of participants found the platform interface to be intuitive, with 67% rating it as “Intuitive” (score of 5) and 33% rating it as “Somewhat Intuitive” (score of 4). These findings suggest that the gamification platform was user-friendly and intuitive, which is an important aspect for successful implementation of gamification in training and educational programs.

The data analysis revealed several key themes related to players’ engagement during gameplay, including motivation, challenge, interaction, and feedback. Players reported being motivated to play the game due to its interactive nature, the challenge of the MCQs, and the opportunity for dialogue with other players. They also highlighted the importance of feedback, both from the game facilitator and from their peers, in enhancing their engagement during gameplay.

Overall, the findings of this qualitative study suggest that the game was successful in engaging players in CT skills development through its motivational and interactive elements. The game’s design, incorporating instructional interventions such as dialogue, authentic problems and examples, and mentorship, aligned with previous research findings and appeared to positively impact players’ engagement during gameplay.

However, it is important to consider the limitations of this conclusion due to the design of the research. Specifically, the findings are based solely on qualitative analysis. To enhance the reliability and validity of the findings, it may be beneficial to employ multiple measures of engagement, such as physiological measurements, direct observation of activities, and various learning analytics, including trace/log data obtained from interactions with the platform.

Notwithstanding its limitations, this study has several implications for practice and further research. It underscores the potential of game-based learning as an effective method for engaging employees in CT skills development in the workplace. The findings also highlight the importance of the research-based instructional design elements, such as dialogue, authentic problems and examples, and mentorship, in enhancing players’ engagement during gameplay. Further research could explore the long-term effects of game-based learning on CT skills development.