Abstract
This qualitative study explores the impact of an educational game on engaging company employees in critical thinking (CT) skills development. The demand for CT skills in the workforce has been highlighted by industry representatives, international organizations, and EU legislation. However, there is a gap between the demand for CT skills and the current abilities of employees, as identified by several studies. To address this gap, game-based learning has been proposed as an effective solution for workplace learning of CT skills. Games provide motivation and engagement in an active learning environment, and research has shown their positive impact on learning outcomes, including CT skills. This study focuses on a game for CT skills development training designed based on instructional interventions identified in previous research. The game was constructed using a gamification platform and players’ engagement during gameplay was explored qualitatively. The paper includes sections on related research, methodology, and results and conclusions.
Keywords
1 Introduction
Surveys of industry representatives [1, 2] as well as reports by international organizations [3, 4] and even EU legislation [5, 6] have highlighted a demand for employees with critical thinking (CT) skills in the workforce. However, despite this demand, several studies conducted by non-governmental organizations [7, 8] have identified CT as one of the biggest current and future gaps in employees’ abilities. The data from these studies suggest that organizations need to focus on developing soft skills like critical thinking [7].
The gap between the demand for CT skills and the number of employees who possess them is a major concern for industry [1, 2, 7, 8]. To address this issue, there is a call for developing effective methods for engaging employees in CT skills learning at the workplace. The development of CT skills is a complex and cognitively demanding learning activity, especially in the context of workplace learning [9].
Games-based learning has been proposed as an effective solution for workplace learning of CT skills, as games provide both motivation and engagement in an active learning environment and have a positive impact on learning goals [10]. Games are often perceived as enjoyable learning tools and contribute to the improvement of critical thinking and reasoning [10]. Previous research has also shown that games can have a beneficial impact on cognitive outcomes, including CT [9, 11].
Based on the demand from industry, the identified lack of CT skills among employees, and the specific aspects of workplace learning, a game for CT skills development training was designed, drawing on instructional interventions identified by Abrami et al. [12]. These interventions include opportunities for dialogue (e.g., discussion), authentic or situated problems and examples, and mentorship, which are integrated into the game design.
To enhance the game’s usability, a gamification platform called Bet-N-Learn (betnlearn.com) was used for constructing the game. Bet-N-Learn is a web-based software that allows for the creation of a game with multiple rounds, each containing a multiple-choice question (MCQ) followed by answer options. Players, who are authorized to access the game via their smartphones or other devices with internet access, can place bets on answer options using game currency (coins). The game facilitator has control over starting and terminating the game, and the platform collects and displays real-time data on players’ results, including bets on answer options, time taken to submit bets, and more, allowing for evaluation of the level of certainty of players.
To gain insights into players’ engagement while playing the game, a qualitative study was conducted to explore players’ experiences. The research question guiding this study was: How do game players describe their engagement when playing the game?
This paper is organized into three sections: related studies (Sect. 2), where examples of research on games used for CT development are provided; the methodology (Sect. 3), where the quality methodology used in the study is explained; and finally, the results and conclusions are presented and discussed (Sect. 4).
2 Related Research
In their systematic literature review, Vlachopoulos and Makri [10] found that engagement is considered a central factor in the majority of game-based learning studies they reviewed. However, research papers focusing on games that develop CT skills often lack qualitative studies on engagement in CT games. For example, Halpern et al. [9] evaluated engagement using quantitative methods, comparing data from tutoring conditions with varying levels of active engagement [9]. Similarly, Stanley and Latimer [13] used questionnaires to assess game enjoyment, which can suggest a correlation between enjoyment and engagement, but may not fully capture the level of engagement during gameplay. Therefore, this study aims to contribute to fulfilling this research gap by examining the qualitative study of engagement in CT games.
3 Methodology
3.1 The Game
The game was designed to be interactive and challenging, with multiple rounds of MCQs that required players to apply CT skills. The game tasks were designed in accordance with the three sections of the Watson-Glaser Critical Thinking Appraisal (WGCTA) [14], specifically evaluation of arguments (EA), recognition of assumptions (RA), and drawing conclusions (DC), while following the guidelines outlined by Morrison and Free [15] and ensuring that the tasks were authentic to the players [12]. WGCTA was chosen because it demonstrates widespread international utilization and enduring longevity, with its first standardized version dating back to 1951 [16]. Now, it has emerged as one of the most commonly employed assessments in business [17].
The game consisted of twelve rounds, each presented as a question (EA), proposition (RA), or statement (DC), with four multiple-choice answers provided for each round. The rounds were grouped into three sets: the first set (S1) and the third set (S3) each contained three MCQs (one EA, one RA, and one DC), while the second set (S2) contained six MCQs (two EA, two RA, and two DC).
The following example is one of the EA MCQ. Question: “Should all the managers in our company have a higher education?” Four options of answer on the choice of the player: (1) “Yes, because we have the financial resources to hire more highly educated workers.” (2) “No, because managers with higher education are a lot more expensive.” (3) “Yes, because highly qualified personnel ensure a high level of productivity in organizations.” (4) “No, because excessive qualification can compromise motivation and so reduce efficiency.”
In the present study, the allocation of coins for wagering on answer options was determined by means of a formula. Specifically, the number of available coins was calculated as the difference between the total number of answer options and one. For example, when four answer options were provided, players had three coins at their disposal. This restriction was implemented with the intention of promoting thoughtful consideration of choices, rather than hasty and indiscriminate wagers on all possible answers.
Russian was chosen for tasks and instruction preparation due to the participants’ proficiency or native speaker status. According to Manalo and Sheppard [18], the language of instruction can impact the ability of individuals to fully demonstrate their critical thinking skills, with poorer performance likely when instructions are given in a foreign language compared to their native language.
3.2 The Training
The CT skills development training using the game was conducted as a standalone intervention for employees of the company. The training comprised five sessions held over a period of five consecutive days, each session spanning three hours. The sessions were designed to explain the relevant concepts of critical thinking (such as deductive reasoning) by providing appropriate examples and addressing the participants’ queries.
During three of the five sessions, the participants engaged in gameplay. They were randomly divided into four groups of three players each and instructed to assume the roles of a company’s board of directors. The groups were then asked to engage in discussions about various important issues that the company faced and make decisions accordingly.
Each group of players engaged in discussions of MCQs in order to identify the correct answer(s). Each player within a group had individual access to the platform and was able to place bets on any of the answer options, regardless of other players’ choices.
After each set of rounds, all training participants convened to discuss the MCQs they had encountered during the game and to evaluate their performance.
Over the course of five days, participants were given three sets of rounds that were administered over three days, with S1 and S3 on the first and third day respectively, and S2 on the second day.
3.3 Design and Participants
The study included 12 adult members of the Management Board of a pet food production company with international headquarters located in Ukraine, representing diverse backgrounds. The training was conducted over the course of five days in April 2021. Prior to administering the survey, verbal consent was obtained from the participants for anonymous usage of the survey data in our research. Written consents were received by mail in May 2021.
3.4 Data Collection
The data collection for this study involved an online survey administered through Google Forms. Participants were instructed to use Incognito mode to ensure anonymity. The survey was administered after the completion of the final training session. Out of the 12 participants, nine feedback forms were received, yielding a response rate of 75%. The survey included various forms of anonymous feedback, such as free-form responses and open-ended questions. Participants were also asked to evaluate how easy it was to interact with the platform interface and how intuitive it was to use, using a Likert-type scale ranging from 1 (Very Difficult) to 5 (Very Easy) and 1 (Not intuitive) to 5 (Intuitive), respectively.
The open-ended questions asked participants to describe how this training differed from others they had attended and what they found to be the most valuable part of the training. All survey questions were presented in Russian, and responses were received in Russian. Later, the responses were translated into English for analysis. We utilized an inductive data analysis approach [19] with the assistance of Atlas.ti software.
4 Results and Conclusions
The participants reported finding the game engaging (e.g., “we got carried away, I was very involved”; “it was interesting and unusual”; “Interactive - dialogues and feedback”). They also reported finding the game interesting (e.g., “Interesting gamification”; “it was very accessible, informative, interesting”).
The participants reported valuable learning outcomes (e.g., "Learn to distinguish where manipulation can be and in what forms"; "an understanding of how ambiguity can distort the real picture and mislead or draw the wrong conclusion"; "Very informative and interesting! Absolutely ‘adjective’ to my work, life situations"). They reported learning about CT, problem-solving, and identifying logical fallacies.
As a part of the survey, all participants were asked if they had experience with similar gaming platforms in other trainings, and none of them reported any prior experience. Moreover, none of the participants had previous experience with the specific gamification platform used in this training.
In order to assess the usability of the gamification platform, participants were asked to evaluate the ease of interaction with the platform’s interface and its intuitiveness. The Likert scale was used, ranging from 1 (Very Difficult/Not intuitive) to 5 (Very Easy/Intuitive). The results showed that the majority of participants found the platform easy to interact with, with 56% rating it as “Very Easy” (score of 5) and 44% rating it as “Easy” (score of 4). Similarly, the majority of participants found the platform interface to be intuitive, with 67% rating it as “Intuitive” (score of 5) and 33% rating it as “Somewhat Intuitive” (score of 4). These findings suggest that the gamification platform was user-friendly and intuitive, which is an important aspect for successful implementation of gamification in training and educational programs.
The data analysis revealed several key themes related to players’ engagement during gameplay, including motivation, challenge, interaction, and feedback. Players reported being motivated to play the game due to its interactive nature, the challenge of the MCQs, and the opportunity for dialogue with other players. They also highlighted the importance of feedback, both from the game facilitator and from their peers, in enhancing their engagement during gameplay.
Overall, the findings of this qualitative study suggest that the game was successful in engaging players in CT skills development through its motivational and interactive elements. The game’s design, incorporating instructional interventions such as dialogue, authentic problems and examples, and mentorship, aligned with previous research findings and appeared to positively impact players’ engagement during gameplay.
However, it is important to consider the limitations of this conclusion due to the design of the research. Specifically, the findings are based solely on qualitative analysis. To enhance the reliability and validity of the findings, it may be beneficial to employ multiple measures of engagement, such as physiological measurements, direct observation of activities, and various learning analytics, including trace/log data obtained from interactions with the platform.
Notwithstanding its limitations, this study has several implications for practice and further research. It underscores the potential of game-based learning as an effective method for engaging employees in CT skills development in the workplace. The findings also highlight the importance of the research-based instructional design elements, such as dialogue, authentic problems and examples, and mentorship, in enhancing players’ engagement during gameplay. Further research could explore the long-term effects of game-based learning on CT skills development.
References
Infosys: Amplifying human potential: Education and skills for the fourth industrial revolution. Infosys (2016)
LinkedIn Talent Solutions: Future of skills. LinkedIn Talent Solutions (2019)
OECD: OECD Skills Outlook 2019: Thriving in a Digital World. OECD (2019). https://doi.org/10.1787/df80bc12-en
World Economic Forum: The Future of Jobs. World Economic Forum (2020)
European Commission: Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions: European Skills Agenda for sustainable competitiveness, social fairness and resilience (COM/2020/274 final) (2020). https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:52020DC0274
The Council of the European Union: COUNCIL RECOMMENDATION of 22 May 2018 on key competences for lifelong learning (2018). https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32018H0604(01)&rid=7
Association for Talent Development: Bridging the Skills Gap: Workforce Development and the Future of Work (2018). https://d22bbllmj4tvv8.cloudfront.net/83/74/450e8cb644188b984d6528d43d58/2018-skills-gap-whitepaper-final-web.pdf
Society for Human Resource Management: THE GLOBAL SKILLS SHORTAGE Bridging the Talent Gap with Education, Training and Sourcing. Society for Human Resource Management, USA (2019)
Halpern, D.F., Millis, K., Graesser, A.C., Butler, H., Forsyth, C., Cai, Z.: Operation ARA: a computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills Creativity 7, 93–100 (2012). https://doi.org/10.1016/j.tsc.2012.03.006
Vlachopoulos, D., Makri, A.: The effect of games and simulations on higher education: a systematic literature review. Int. J. Educ. Technol. High. Educ. 14(1), 1–33 (2017). https://doi.org/10.1186/s41239-017-0062-1
Watkins, M., Beckem II, J.M.: Bringing life to learning: immersive experiential learning simulations for online and blended courses. OLJ. 16, 61–70(2012). https://doi.org/10.24059/olj.v16i5.287
Abrami, P.C., Bernard, R.M., Borokhovski, E., Waddington, D.I., Wade, C.A., Persson, T.: Strategies for teaching students to think critically: a meta-analysis. Rev. Educ. Res. 85, 275–314 (2015). https://doi.org/10.3102/0034654314551063
Stanley, D., Latimer, K.: ‘The ward’: a simulation game for nursing students. Nurse Educ. Pract. 11, 20–25 (2011). https://doi.org/10.1016/j.nepr.2010.05.010
Watson, G., Glaser, E.M.: Watson-Glaser Critical Thinking Appraisal (1980)
Morrison, S., Free, K.W.: writing multiple-choice test items that promote and measure critical thinking. J Nurs Educ. 40, 17–24 (2001). https://doi.org/10.3928/0148-4834-20010101-06
Bernard, R.M., Zhang, D., Abrami, P.C., Sicoly, F., Borokhovski, E., Surkes, M.A.: Exploring the structure of the Watson-Glaser critical thinking appraisal: one scale or many subscales? Thinking Skills Creativity. 3, 15–22 (2008). https://doi.org/10.1016/j.tsc.2007.11.001
Possin, K.: Critique of the Watson-Glaser critical thinking appraisal test: the more you know, the lower your score. IL. 34, 393 (2014). https://doi.org/10.22329/il.v34i4.4141
Manalo, E., Sheppard, C.: How might language affect critical thinking performance? Thinking Skills Creativity. 21, 41–49 (2016). https://doi.org/10.1016/j.tsc.2016.05.005
Bingham, A.J., Witkowsky, P.: Deductive and inductive approaches to qualitative data analysis. In: Vanover, C., Mihas, P., and Saldaña, J. (eds.) Analyzing and Interpreting Qualitative Research: after the Interview, pp. 133–146. SAGE, Los Angeles (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mets, J., Bauters, M. (2023). Engaging Company Employees in Critical Thinking Through Game-Based Learning: A Qualitative Study. In: Huang, YM., Rocha, T. (eds) Innovative Technologies and Learning. ICITL 2023. Lecture Notes in Computer Science, vol 14099. Springer, Cham. https://doi.org/10.1007/978-3-031-40113-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-40113-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40112-1
Online ISBN: 978-3-031-40113-8
eBook Packages: Computer ScienceComputer Science (R0)