Abstract
Virtual simulations (VS) are gaining more popularity in vocational education and training (VET) to train professional competencies. However, implementing VS into organisations requires the involvement of trainers in, and the development of their knowledge to fulfil a meaningful approach to integrating this technology into teaching. In the Estonian Academy of Security Sciences (EASS) VS have been used for more than a decade to train and assess rescue and police staff work and related competencies such as decision-making and problem-solving. As there is a lack of research as to the role of trainers in the implementation process of virtual simulation-based training (VSBT) in organisations, a survey among EASS trainers (N = 146) was used to reveal their role in the implementation, their attitudes towards the use of VS and their Technological Pedagogical Content Knowledge (TPACK). While attitudes grow more positive with the use of VS and especially with training others to use it, integrated knowledge about using technology in training seems to depend on the active participation of trainers in its creation. We discuss the important role of involving trainers in the implementation of VS-based training to ensure the successful use of VS in vocational education.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Introduction
As computer technology grows more powerful, it is now feasible to create computer-based simulations in many domains (e.g. healthcare (Keskitalo, 2022), police (Sjöberg, 2014) and rescue incident commanders’ training (Lamb et al., 2021)). Technologies used to create virtual environments include virtual reality, augmented reality or simply desktop computers or mobile phones that present virtual simulation scenarios (Kostov & Wolfartsberger, 2022). Because of the different technologies used to create virtual simulation scenarios, no unified definition is agreed on cross-disciplinary (Foronda et al., 2020; Kardong-Edgren et al., 2019). In this study, virtual simulations (VS) are defined as scenarios or problem tasks that are presented using virtual reality (software and hardware) to imitate processes or activities related to working life with the aim of developing the trainees’ competencies (Polikarpus & Danilas, 2021). For example, a trainee might encounter a fire scene and needs to decide on a course of action. The actions of he/she in the VS have an influence on the course of the VS scenario, e.g., actions taken (firehose versus fire extinguisher) contribute to stopping the fire (Polikarpus & Ley, 2021). In VET, VS can be used to train dynamic decision-making competencies (Reis & Neves, 2020) or to gain awareness of multi-professional collaboration (Prasolova-Forland et al., 2017). Nonetheless, the integration of virtual simulation-based training (VSBT) into VET is a complex process, and future research should examine efficient design solutions in virtual reality environments (Ke et al., 2020).
Even if VS have been shown to be an effective training tool (Sapiano et al., 2018), most of the current research on VSBT reports the use of VS that has been created for a specific training purpose by experts using standard scenarios. Research studies have been conducted using in-depth expert evaluations (Drakos et al., 2021) or organised as one-off training sessions for specific target groups in a controlled setting or for research purposes (Kincaid, 2015; Sapiano et al., 2018). While such an approach might demonstrate effectiveness in specific settings, it does not give useful guidance on how to create VSBT for specific training contexts in organisational settings in a systematic and sustainable way that would create meaningful learning opportunities for trainees and trainers.
In the organisational context, trainers have a central role in facilitating VSBT (Keskitalo, 2022), as they need to be aware of the capabilities of the technology (Kostov & Wolfartsberger, 2022), need to choose or build these VS for training and eventually integrate these into their training approach. A factor that has limited the wide use of VSBT in training organisations is the need for skilled trainers (Küttner-Magalhães & Libânio, 2022). Especially in VET, research on trainers’ digital competence is still limited, and their attitude towards the use of technology and other contextual factors are important research directions to tackle (Antonietti et al., 2022; Cattaneo et al., 2022).
As there is a scarcity of research on the conditions of the large-scale use of VS in authentic organisational training context focusing on the trainers’ role in the implementation of VSBT, we look at a case of a public Estonian VET provider in which VSBT has been promoted as part of an organisational strategy. The Estonian Academy of Security Sciences (EASS) is the only internal security education training institution in Estonia that provides vocational and continuous training courses (EASS, 2022). It is an institution of professional higher education, characterised by its specialisation to professional qualifications in the fields of police and border guards, rescue, taxation, customs, and corrections at European Qualifications Framework (EQF) levels 4 to 7 (Valk & Kratovitš, 2021). In the EASS, the VSBT has been used in various forms and several named fields for more than twelve years and has therefore developed systematic strategies to integrate it into training practices (Põder et al., 2015; Polikarpus et al., 2020). Due to these circumstances, the case of EASS presents an opportunity to examine the trainers’ role in implementing this type of training.
To implement VSBT in VET, trainers should integrate their technological, pedagogical, and content knowledge with their knowledge of a training institute context (Mishra, 2019). The context in its various levels (micro, meso, macro) should be considered during the studies of trainers’ integrated knowledge (Rosenberg & Koehler, 2015; Schmid et al., 2020). Therefore, in order to implement VSBT into an organisation, the role of trainers needs to be better understood. In this study, we aimed to find out the role of trainers in the implementation process of VSBT in an organisation and how this role is related to the trainers’ attitude towards the use of VS and their Technological Pedagogical Content Knowledge (TPACK). We reported the results of a large-scale survey among trainers at an institution of professional higher education to explore their role in the implementation process of VSBT.
The Role of Trainers in the Use of Virtual Simulations for Training in Vocational and Professional Education
For VSBT to be implemented in an organisation, VS has to be created (Polikarpus & Ley, 2021) or existing ones used (Bourdeau et al., 2021). In both cases, skilled trainers play a key role in effectively implementing VS in training organisations (Keskitalo, 2022). Accordingly, we also focused especially on trainers’ attitudes towards using VSBT and their integrated knowledge of using technologies in training.
Trainers’ Attitude Towards Technology
Practising trainers’ attitudes towards virtual reality technology and the feedback that they provide is an important factor in implementing VSBT (Pedram et al., 2020). As a long history of research into the acceptance of technology shows, a positive attitude towards a new technology has direct effects on the intention to use it (Luik & Taimalu, 2021; Teo et al., 2017), especially in the voluntary context (Kemp et al., 2019; López-Bonilla & López-Bonilla, 2011). The reasons could be that trainers with a positive attitude are readier to experiment with technologies and discover certain uses for VET.
Most studies on the attitude of teachers towards technology use have been conducted in the general education domain (Luik et al., 2019; Teo et al., 2017) and address teacher education students (Luik & Taimalu, 2021; Yerdelen-Damar et al., 2017). The generalisability of VET targeting adult learners needs to be clarified. Moreover, the generalisability of studies that look at the intention to use technology to the actual use of it in VET needs to be better understood. Even more, large-scale studies that could paint a more realistic picture of innovative technology use in VET on macro meaning national or meso meaning organisational level are rare. An exception is a recent studies among Swiss VET trainers (Antonietti et al., 2022; Cattaneo et al., 2022), who found that for practising trainers, attitudes are associated with their knowledge of how to use technology in training (as measured by TPACK).
With regard to the use of VS by trainers in VET, there is even less evidence in the current literature. Some studies exist that look at teacher attitudes towards the use of virtual reality (Pedram et al., 2020) or serious games (Heldal et al., 2016), but these seem to be rather experimental uses of approaches or dealing with implementation challenges. Therefore, we decided to measure working trainers’ attitudes towards VS use in EASS in this paper, referred to as AVSU.
Trainers’ Integrated Knowledge for Using Technology in Training
Studies among pre-service teachers highlight the necessity for theoretical knowledge and practice to be developed for technology integration in education (Altun, 2019). Trainers using VSBT need specific knowledge and skills to use technology. TPACK framework is widely applied to measure teachers’ knowledge related to the integration of technologies into teaching (Mishra, 2019; Yeh et al., 2021). TPACK allows to measure the Pedagogical Knowledge (PK), Content Knowledge (CK), and Technological Knowledge (TK) but focuses especially on the integration of these components Pedagogical Content Knowledge (PCK), Technological Pedagogical Knowledge (TPK), Technological Content Knowledge (TCK), and Technological Pedagogical and Content Knowledge (TPCK) (Koehler & Mishra, 2008; Schmid et al., 2020). In this article, we use the term “non-integrated knowledge” to refer to TK, PK, and CK components and “integrated knowledge” to refer to TPK, TCK, PCK and TPCK components. We use TPACK to talk about all seven dimensions.
For named TPACK components, numerous self-report measurement instruments have been proposed, e.g. by Pamuk et al. (2015) and Schmid et al. (2020). These have been widely applied to measure teachers’ knowledge integration in different domains, such as pre-service chemistry teachers (Deng et al., 2017), mathematic teachers (Yan et al., 2018) or measuring across different domains (Luik et al., 2019). Intriguing research that left out the use of technology in teaching and focused only on economics CK and PCK knowledge found that regardless of trainers’ academic or practical preparation, they had similar CK and PCK (Holtsch et al., 2019). Therefore, we studied all trainers in EASS irrespective of their work experience or employment type.
In VET, however, TPACK is needs to be researched more. Therefore, it is important to find out how personal factors like TPACK knowledge and attitude are related to VET teachers’ digital competencies (Antonietti et al., 2022; Cattaneo et al., 2022). Because the higher the trainers’ integrated knowledge is, compared to non-integrated, the more meaningful and useful knowledge it becomes to make the best use of the possibilities of the technology to support trainees learning (Schmid et al., 2020). As previous studies focus on pre-service teachers’ intended use of new technologies in the context of Estonia (Luik & Taimalu, 2021), it is necessary to look into TPACK knowledge of practising trainers and the relation to patterns of actual use of technology in a training institute – meso level context (Rosenberg & Koehler, 2015) in Estonia. Furthermore, general education in-service teachers’ TK has been found to be of the lowest rank construct followed by TPK (Luik et al., 2019), indicating there might be challenges related to implementing VSBT into training in Estonia.
Participation of Trainers in the Implementation of Virtual Simulation-based Training
Research suggests that contextual factors, such as the amount of organisational leadership or co-workers’ support adopting a technology, influence teachers’ personal adoption decisions (Harris & Hofer, 2011; Martín-García et al., 2019). For adopting technology-enhanced instruction, a factor that seems especially important is to involve teachers actively in the co-design of learning methods and scenarios (Ley et al., 2021). Therefore, it can be assumed that an important dimension of the VSBT implementation in the authentic training organisation context is how much trainers participate in the process.
With regards to development of teachers’ TPACK, “Learning by design” is used, which means they need to map technology and tool(s) affordances to learners’ needs, pedagogical concerns, and specific topics and implement them in a meaningful way into authentic teaching contexts (Yeh et al., 2021). Also, the collaborative design approach to create tools for inquiry-based mathematics learning found that teachers’ PCK plays an important role in the design of pedagogically sound digital tools, and teachers develop their TPK through the implementation of collaborative design of teaching materials (Yan et al., 2018). A collaboration-enriched TPACK framework indicates a theoretical relationship between collective TPACK and personal TPACK in the organisational context (Yeh et al., 2021). Therefore, it could be assumed that through collective processes like co-design, teachers develop collective TPACK, which probably enhances their personal TPACK and vice versa (Yeh et al., 2021).
Here we focus on the role of trainers in implementing VSBT in an organisational context. This is especially relevant for VSBT, as virtual reality software products differ in how much trainers can design the VS scenarios they will use later in training. While some products are shipped as a ready-made training package (with fixed scenarios included), others allow co-design of the scenarios by the trainers. Differences in this type of VS implementation in the concrete context and its relation to trainers’ personal factors, AVSU and TPACK, have not yet been empirically researched.
We conducted our study in EASS to measure the working trainers’ personal TPACK. Because earlier studies in the same organisational context show that trainers can participate in co-designing VS in EASS (e.g. by using The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021)), we assume that this might lead to the development of collective TPACK (Yeh et al., 2021) in EASS. Trainers co-authoring of VS using different theoretical approaches to train situational awareness (Polikarpus & Ley, 2021; Polikarpus et al., 2022) has led to the implementation process of VSBT in EASS (Polikarpus & Danilas, 2021). Therefore, there is a widespread use of VSBT highlighted as an innovative aspect of the EASS approach to teaching (Bryant et al., 2019). In consequence, EASS provides a good testbed for examining the different roles of trainers in the implementation process of VSBT.
Based on research regarding teachers´ acceptance and integration of new technologies into their own teaching (Eutsler & Long, 2021; Martín-García et al., 2019; Olson et al., 2020; Schmidt et al., 2009) and our EASS context knowledge, we indicated six different groups of trainers role in implementing VSBT in EASS in Table 1. The first three groups as variations of non-use and three of use (Olson et al., 2020) VSBT.
Research Problem and Hypotheses
Results of earlier TPACK studies in general education indicate the relationships among teachers’ attitudes towards the use of technology, TPACK, and their plans to carry out technology-integrated lessons (Deng et al., 2017; Luik & Taimalu, 2021). The two most dominant determinants of behavioural intention were TPACK and attitudes towards computer use (Teo et al., 2017). Based on the discussion above, we claim that there is a gap in research on the role of trainers in VSBT implementations in training institutions. Hence, we aim to answer the following research question: What is the role of trainers in the implementation process of VSBT in an organisation and how is the role of implementation related to the trainers’ attitude towards the use of VS and TPACK components?
Given that participation in creating technology-enhanced learning approaches (e.g. "Learning by design") by trainers is an important factor (Yeh et al., 2021), we assume that this will play a crucial role in integrating knowledge, and it will positively influence AVSU. Based on the previous research (Rosenberg & Koehler, 2015) which shows that TPACK is developed in specific context, we believe that the contextual micro, meso and macro level conditions trainers work in when implementing VSBT add to their positive AVSU and integrated knowledge (specially TPK, TCK and TPCK), mutually reinforcing each other. Based on earlier used educational technologies acceptance models (Kemp et al., 2019) we assumed that a meaningful positive attitude contributed to the interest in experimenting with VS. Furthermore, participation of trainers through taking an active role in implementing VSBT (VS user groups 4–6) could eventually contribute to their integrated TPACK knowledge (Yeh et al., 2021). Then using VSBT and especially training trainers to use VS contributes to a further positive attitude towards VSBT. Higher AVSU should help to get started using VS (Sarıtaş, 2015), while participation in the creation of VS helps trainers to integrate their knowledge (Yeh et al., 2021). We illustrate the expected relations between named constructs in Fig. 1.
This has motivated the following hypotheses:
-
H1: Trainers who have created VS (group 5) and who have trained trainers to create VS (group 6) have a more positive AVSU than those who have just used VSBT (group 4) or only tested (group 3) or know about it (group 2).
-
H2: Trainers who have created VS (group 5) and trained trainers to create VS (group 6) have more integrated TPACK knowledge than trainers who have only used VS (group 4) as a ready-made simulation package in their training.
Method
In order to test these hypotheses, a survey study was conducted among 146 trainers of EASS. Respondents had various roles in using VS in their training (independent variable), including active use and creation of VS scenarios. A questionnaire was developed that measured two main dependent variables, trainers AVSU as well as their self-assessment on TPACK.
Sample and Research Context
EASS is the biggest higher-level education state-owned institution under the Estonian Interior Ministry, where current and future staff members of civil servants are trained in four colleges at different loctions in Estonia (Bryant et al., 2019). There are around 1000 trainees’ studying educational curriculums, and in addition, approximately 5000 trainees get continuing yearly training courses in EASS, and they are taught by 85 full-time academic staff members and around 200 part-time trainers (Sisekaitseakadeemia, 2018). It is stated in the international institutional accreditation decision, “The self-improvement, development and evaluation of the academic staff are comprehensive and systematic. They also support the further development of teaching methods, research opportunities and curricula.” and “Teachers are motivated and qualified.” (Otto & Bauman, 2019, p. 5).
The study was conducted among full-time and part-time trainers in EASS. Participation in the survey was voluntary and anonymous. We used the open-source survey software tool LimeSurvey (LimeSurvey GmbH, 2021) and a web-based link was shared through e-mail lists. We downloaded survey participants’ answers from LimeSurvey and excluded the responses from those who had not completed the VS user group question (see Appendix).
Altogether 146 answers were received that could be used for analyses, meaning approximately 51% of all trainers of the organisation responded to the survey. Among survey participants were 50 (34%) full-time trainers, 63 (43%) part-time trainers, 12 (8%) from management and 8 (6%) from support staff with teaching responsibilities, 13 (9%) gave other option answers or did not give their position. The main field of teaching for 46 trainers (32%) was rescue, 38 police and border guard trainers (26%), correction nine trainers (6%), tax and customs seven trainers (5%) and 46 trainers were teaching in different fields or did not answer the question. The average age of participants was 44.9 years (SD 9.1). The youngest trainer was 21, and the oldest was 79. 54% were men (79 male instructors) and 59 females (40%), and eight trainers (6%) preferred not to give their gender.
Concerning trainers’ preparation to teach in EASS, it turned out that 32 trainers out of 146 have an occupational qualification certificate as an andragogic, vocational teacher or other similar trainer professional certificates, 70 have participated in teaching methods training in EASS and 64 in similar training at other institutes. However, 31 participants did not answer the question about their preparation for teaching.
Measures and Instruments
The self-report questionnaire had three parts (see Appendix).
Trainers’ attitude towards VS use – AVSU
For the current study, we adopted items typically used for measuring the construct “attitudes towards technology” in studies using the Technology Acceptance Model (e.g.López-Bonilla & López-Bonilla, 2011; Teo et al., 2017). Specifically, we employed a subscale consisting of four items that had already been translated into the Estonian language and validated by Luik and Taimalu (2021) using a confirmatory factor analysis with standardised factor loadings ranging from 0.690 to 0.826, and scale reliability Cronbach’s alpha of 0.85.
The items were adapted for the current study by replacing “technology” with “Virtual Simulations”. For example, the first item translation into English was “Nowadays it is important to use virtual simulations in training.” All items were rated on a five-point Likert scale (1 “strongly disagree” to 5 “strongly agree”), and all four items are translated into English in Appendix part II.
Trainers’ personal TPACK
We adapted and translated the TPACK-XS instrument, originally developed by Schmid et al. (2020) and validated in a sample of Swiss pre-service upper secondary school teachers (N = 117). A confirmatory factor analysis in that original study had confirmed the factor structure and reliability of the scales. For the present study, all items of TPACK sub-scales were used. They were translated from English into Estonian by the first author and translated back into English by an independent researcher, and the two English versions were then compared and inconsistencies rectified. After that, one Estonian-speaking person made sure terms like “teaching style” and “technology” were always used in the same wording in the Estonian language. The amendments were made only to the introductory text of the original instrument. For example, the phrase “to the subject in which you have written the lesson plan” was replaced with the phrase “module that you teach in the EASS”. All items can be found in the Appendix. The term VS was not used in the first part of the survey.
The translated instrument consisted of the seven TPACK subscales. Each subscale measured with four items in the first part of the survey: 1) Pedagogical Knowledge (PK), 2) Content Knowledge (CK), 3) Technological Knowledge (TK), 4) Pedagogical Content Knowledge (PCK), 5) Technological Pedagogical Knowledge (TPK), 6) Technological Content Knowledge (TCK), and 7) Technological Pedagogical and Content Knowledge (TPCK). All items were rated on a five-point Likert scale (1 “strongly disagree” to 5 “strongly agree”).
VS user Groups
The second part of the survey contained items to establish the independent variable VS user group in line with the grouping introduced in Table 1. First, a definition of VS was provided (see Appendix), followed by the question, “Which of the following statements best describes your experiences with VS so far”. Six statements were used that allowed to creation of an ordinal scale of VS user groups. Mainly to ensure that VS user group questions fit our research context, we conducted several steps to ensure the reliability and validity of our survey instrument (more in chapter Testing Reliability and Validity of the Survey Instrument). First, the survey was piloted, by using a think-aloud protocol (Wolcott & Lobczowski, 2021) with six different experts to make sure the wording of items was clear for VS users and non-users.
Other measures
In the third part, questions were asked about the characteristics of the sample and personal data like respondents’ age, gender, a field of teaching, pedagogical preparation and their role in the organisation. The order of questions in the survey was changed after piloting it with experts, and used VS definition was given in part two prior to the VS user groups question (see Appendix).
Data Analysis
The first dependent variable AVSU was calculated as a mean score from four items (see Appendix). For the second dependent variable (integrated knowledge), we first derived mean scores from each of the seven TPACK subscales (four items each). Then we computed the mean value of the “non-integrated knowledge” from subscales CK, TK, and PK. For “integrated knowledge”, we used the four subscales (PCK, TPK, TCK, TPCK) and computed a measure for “integrated knowledge” as a mean value for each person.
The independent variable (VS user group) was created based on the item “Which of the following best describes your experience with virtual simulations so far?” (Appendix part II). Trainers were grouped based on their “highest order” answers to this question. For example, if trainers answered that “As a trainer, I use virtual simulations in my training” (VS user group 4) and “I myself create virtual simulations for training.” (VS user group 5), then he/she was added to group 5. In the first VS user group (response no 1: “Do not know any VS”) were 34 trainers, and the group were named “No VS info”. The second group (response 2, “Knows one or more VS”) also had 34 trainers and was named “Knows VS”. These were the two biggest groups, so there are 68 trainers (46.6%) who had not used any VSBT as trainers or trainees. The next response was group 3, “Has tested VS” comprised 30 trainers (20.5%) and were also considered non-users of VSBT. In our research, we especially focused on groups four to six. VS user group 4 (15 trainers) named “Has used VS”, group 5 (17 trainers) “Has created VS”, and user group 6 (16 trainers) “Has trained trainers”. All these 48 trainers (32.9%) have implemented VS in their teaching, so they are active VSBT users.
We validated the survey instrument and calculated descriptive statistics for all VS user groups (means, standard deviation, and Skewness and Kurtosis to check for normality). For H1, we conducted a one-way ANOVA (VS user groups as an independent and AVSU as a dependent variable) and a Levene’s test to check the homogeneity of variance and a Dunn’s Post Hoc test. For H2, we conducted a two-way repeated measure ANOVA, where the first within the subject factor was integrated knowledge (integrated vs non-integrated), and the second between the subject factor was VS user groups. We used paired T-test for post hoc analyses. Some of the analyses was done for the whole sample, and some were done only for the groups of trainers actively using VSBT (groups 4–6). These analyses were conducted using Jasp version 0.16.1 (JASP Team, 2022).
Testing Reliability and Validity of the Survey Instrument
We used the responses gathered from our sample (N = 146) to check the reliability and validity of the instrument used: we applied some general measures for it. We calculated Cronbach alpha for each TPACK subscale and AVSU as a measure of internal consistency and used correlation coefficients between survey subscales (see Table 2) to check for the construct validity of the instrument.
Table 2 gives an impression of the reliability (internal consistency) by Cronbach alpha, a measure of the intercorrelation of all items in one scale. Given that all Cronbach alpha scores are greater than 0.8, this indicates a highly reliable instrument. In addition, Table 2 presents intercorrelations between the constructs indicating convergent and discriminant validity of the scales as follows: Concerning the intercorrelations between constructs; we expect that these would be significantly lower than the Cronbach alpha scores. As can be seen in Table 2, this assumption is met, as Pearson’s r generally lies between 0.2 and 0.6, with most of them around 0.4. The overall patterns also seem to confirm the idea of TPACK in those correlations of integrated knowledge domains (e.g. PCK) are higher with their respective subparts (e.g. PK and CK) than with the unrelated domain (e.g. TK). Intercorrelations between knowledge (TPACK components) and attitude scale (AVSU) are small to medium for technology-related knowledge (like TPK, TCK, TPCK), confirming prior research on the positive link between technology attitudes and knowledge. For the other TPACK components, correlations with attitude were non-existent.
Table 3 gives further descriptive results for TPACK components and AVSU for all participants of the survey. The trainers have self-assessed the content knowledge (CK) as the highest and the technology knowledge (TK) as the lowest. The standard deviation was highest (SD = 0.83) on items connected to AVSU and lowest on CK (SD = 0.58). We also report in Table 3, PK and integrated PCK, TPK, TCK, TPCK knowledge means and minimum and maximum scores.
From correlation studies and descriptive statistics, we conclude that these results provide conclusive evidence of a reliable instrument with sufficient construct validity. We can conclude that the transfer of the instrument to this new context in vocational and professional education was successful.
Results
Trainers’ Attitude Towards Virtual Simulation Use (AVSU)
To illustrate the first hypothesis, trainers who have created VS (VS user group 5) and who have trained trainers to create VS (group 6) have a more positive AVSU than those who have just used VSBT (group 4) or only tested (group 3) or know about it (group 2), Fig. 2 shows means and SD for AVSU for each VS user group.
Generally, AVSU mean values rise for trainers who have used VS in some way (groups 3 to 6) towards more proficient use of VS. It is also seen that those who have trained trainers to build VS have the highest self-assessment for AVSU statements (M = 4.47) and the lowest standard deviation SD = 0.5. Trainers, who have not had any contact with VSBT (groups 1 and 2) had lowest AVSU means (M = 3.68; group 1 SD = 0.81 and group 2 SD = 0.85).
To check whether AVSU varied significantly between VS user groups, a one-way ANOVA was performed. None of the prerequisites was compromised (skewness and kurtosis were in acceptable ranges, and the normality assumption was complied with). The ANOVA resulted in statistically significant differences in self-assessed AVSU between the VS user groups (F = 2.66; p = 0.03; df = 5), although the effect size was small (η2 = 0.09 and ω2 = 0.05). A non-parametric Post Hoc test (Dunn) used for testing small sub-sets of pairs (Goss-Sampson, 2020) showed that only VS user group 6 “Has trained trainers” had statistically significantly higher self-assessments for AVSU than all the other groups (p = < 0.001).
Trainers’ TPACK knowledge
To test the second hypothesis, “Trainers who have created VS (VS user group 5) and trained trainers to create VS (group 6) have more integrated TPACK knowledge than trainers who have only used VS (group 4) as a ready-made simulation package in their training.”, we compared each trainer’s “non-integrated” and “integrated knowledge” in their VS user group to which they belonged. Table 4 shows that across all groups, trainers rated their integrated knowledge lower than non-integrated knowledge.
As we were especially interested in trainers using VSBT, we restricted the following analysis to only those trainers who were actively using VS in their training (user groups 4–6). In Fig. 3, it is visible that group 4, “Has used VS”, has self-assessed their non-integrated knowledge higher than integrated knowledge, while those differences are smaller for group 5, “Has created VS”, and 6, “Has trained trainers”. To confirm this pattern statistically, we conducted a two-way repeated measure ANOVA with VS user groups 4–6 as a between-subjects factor and integrated knowledge (non-integrated vs integrated) as a within-subject factor.
The ANOVA for VS user groups 4–6 resulted in significant within subjects’ main effects (non-integrated vs integrated knowledge) (F(1) = 7.46; p = 0.01), meaning that across all conditions, trainers were more likely to self-assess their integrated knowledge as lower than the knowledge in the non-integrated components. This is in line with the assumptions of TPACK, where it is assumed that integration of knowledge requires additional explicit learning and knowledge construction processes. The ANOVA also resulted in a significant interaction (integrated knowledge x VS user groups) (F(2) = 3.31; p = 0.05). However, the between-subject effect (VS user groups) had no statistically significant result (F(2) = 1.25; p = 0,3). The assumption of sphericity was not compromised.
To compare trainers’ integrated and non-integrated knowledge self-assessments paired T-test as a post hoc test was used. Because the authoring of VS needs integrated knowledge, we assumed that trainers who have just used but not created VS themselves have statistically significant differences between their self-assessments of non-integrated and integrated knowledge and trainers who have created VS themselves do not. If we compared user group 4 “Has used VS” non-integrated and integrated self-assessments (t = 3.70; p = 0.002; df = 14; Cohen’s d = 0.96), we found a statistically significant difference. However this is not the case for group 5, “Has created VS” (t = 0.08; p = 0.94; df = 16; Cohen’s d = 0.02), and neither for group 6, “Has trained trainers” to use VS (t = 0.88; p = 0.39; df = 15; Cohen’s d = 0.22).
To conclude, the trainers who had used VSBT judged their integrated knowledge lower than their non-integrated knowledge. This was not the case for those trainers who had participated in the creation of scenarios or those who had instructed others. This is in line with our assumption that involving trainers actively in the creation of VS scenarios improves, especially their self-assessment of integrated knowledge.
There is a further trend visible in Fig. 3, namely that group 6, “Has trained trainers”, to use VS seemed to assess their knowledge lower than group 5, “Has created VS”, which seems counterintuitive. However, as the ANOVA did not confirm this effect of a downward trend statistically, it needs to be left to future research to confirm this trend.
Discussion
In this paper, we researched the implementation of VSBT in VET from three perspectives: 1) trainers’ attitude (measured as AVSU) and 2) technology pedagogy and content knowledge (measured as TPACK components), and 3) a trainers’ role in the implementation process of VSBT (measured as six VS user groups). Earlier research found that the attitude towards technology, among other personal factors, contributes to teachers´ digital competence development (Cattaneo et al., 2022). It is clear that a positive attitude towards technology encourages its adoption (Ertmer et al., 2012), but as far as we know, attitudes towards the use of technology have not been researched in the context of VSBT in VET and with regards to its relation to in-service trainers using VSBT.
Also, several studies about TPACK have been carried out among pre-service teachers (Luik & Taimalu, 2021; Schmid et al., 2020; Teo et al., 2017), but the strength and uniqueness of the current study are that it was carried out among vocational trainers in an institution of professional security education. Instead of intentions to use technology in teaching, we researched training practices using VSBT in its specific context. This allowed us to get a more realistic picture of how AVSU and TPACK are related to the actual use of VSBT in a large sample of VET trainers. However, looking at specific groups of users revealed some interesting dynamics in terms of knowledge and attitude.
While self-assessed TPACK knowledge seemed to be rather high throughout the VS user groups, TK was rated the lowest in our sample (except group 1, where it ranked sixth), confirming prior research among in-service teachers in general education in Estonia (Luik et al., 2019). This suggests that integrating new technologies into teaching can be arduous. The highest-ranked non-integrated knowledge component was CK in all VS user groups except group 6, which ranked highest in their PK. If we compared the order of ranking between VS user groups, we noticed that groups 2–4 had exactly the same ranking even if groups 1–3 were not using VSBT and group 4 trainers used VSBT as facilitators. This trend might indicate that being in the role of facilitator of VSBT might not effectively develop trainers’ integrated knowledge. VSBT user groups 5 and 6 rankings were different from non-users of VSBT and group 4 as well as from Estonian general education in-service teachers (Luik et al., 2019). It could be that differences in ranking indicate that trainers who create VS and train others to use them are more involved in collaborative design (Yan et al., 2018) and therefore develop stronger collective TPACK (Yeh et al., 2021).
This study provided meso-level overview of the operationalisations of TPACK (Rosenberg & Koehler, 2015) in EASS. Group 4 (15 trainers) are trainers who act as facilitators of VSBT, and although the importance of their pedagogical practice is highlighted in earlier research (Keskitalo, 2022), we did not find out any meaningful differences in their personal TPACK compared to trainers who do not use VSBT. At the same time, VS user groups 5 and 6 (altogether 33 trainers) probably have developed collective TPACK by taking part in an iterative technology-mapping process, considering tool affordances, trainees’ needs, and pedagogical matters, taking into account important concepts of specific topics in a meaningful way for realisation in the authentic context (Yeh et al., 2021).
We found out that our hypothesis one (H1) was supported. Trainers who use VS in their teaching (especially VS user groups 4–6) have a more positive attitude towards it compared to the trainers who do not use VS in their work. Moreover, trainers who train trainers (user group 6) in the use of VS have a more positive attitude than those using it to facilitate VSBT (group 4). This confirms results from earlier studies that a positive attitude towards the use of technology is related to the intention to use technology and in training others (Luik & Taimalu, 2021). Organisations that intend to boost the use of VS could therefore identify those trainers who already have a positive attitude towards the use of technology and convince them to help others to create suitable learning scenarios for VET VSBT. This approach could be effective because trainers already know the organisation at the micro and meso level (Rosenberg & Koehler, 2015).
Our hypothesis two (H2) was partially supported, as the patterns of results on TPACK showed some interesting differential effects. Specifically, not all TPACK components showed a similar growth from VS user groups 1 to 6, but it was the integrated knowledge that differentiated especially groups 5 and 6 from the rest. Generally, trainers across the user groups judged their integrated knowledge statistically significantly lower than their non-integrated knowledge (within subjects, the main effects of non-integrated vs integrated knowledge was F(1) = 7.46; p = 0.01). It is in line with the ideas in TPACK that integration of knowledge is an additional step to constructing meaningful technology application knowledge and that high self-assessment scores in technology, content and pedagogical knowledge are not by themselves sufficient to ensure meaningful integration of technology into teaching. Trainers learn technology by VS design. However, “design is not something that can be taught by lectures and demonstrations” (Koehler & Mishra, 2005, p. 98). A key finding (as evidenced by the significant interaction effect) is that this difference between integrated and non-integrated knowledge is not found in trainers who create VS scenarios or those who instruct others to do so (VS user groups 5 and 6). We claim that this is evidence that these two groups have developed more integrated knowledge than those who are just using a ready-made training package in their VSBT (VS user group 4). Furthermore, it can be expected that the higher the trainers’ TPK, TCK and PCK are, as compared to TK, PK, and CK, the more meaningful and useful knowledge integration occurs (Schmid et al., 2020), which also would lead to a more meaningful application of those technologies into training. Moreover, both positive AVSU and the development of integrated knowledge are related to the trainers’ role in using virtual reality technology and the creation of VS scenarios. This finding is similar to an earlier study that the use of computers in teaching TPACK and attitude was the biggest influence (Teo et al., 2017).
We also noted a somewhat unexpected result, as descriptively, the group that has trained trainers to use VS (user group 6) tended to judge their knowledge generally lower than those trainers who create VS (group 5, see Table 4). Probably the group of “train the trainers” were evaluating themselves more critically in self-assessment than those who don’t train others to use innovative technology in training. Also, it appears that their lower ratings are especially marked in knowledge relating to content knowledge (CK). It seems that if a trainer needs to give advice to other instructors from his/her own subject field, they self-assess content knowledge lower than pedagogical knowledge. Similarly, user group 5, “Has created VS”, self-assessed their CK as higher than PK. As the ANOVA could not confirm this trend, these interpretations remain speculative for now and should be verified in a larger sample or with more in-depth qualitative studies in one organisation to keep the context of using VSBT as an unchangeable variable.
Obviously, context and organisational support mechanisms are also important prerequisites for the effective implementation of VSBT. Before the use of The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021), the research in EASS stated that there were not enough trainers to implement VSBT (Põder et al., 2015). Our research showed that 23,3% of trainers in EASS have not heard about the possibility of implementing VSBT in EASS. We also found out that 67% of respondents in the survey were not using VSBT, while 48 (33%) were using it. While we have not focused on context related factors in detail in this research, there is some evidence that organisational context has also had an impact on our results. For example, we found that among VS user groups 4 to 6, there are more trainers from the rescue and police college compared to other colleges, centres, or institutes in EASS. Research about VS and serious games use in different countries’ emergency management organisations found that VSBT is seen as a successful way of training vocational competencies (Heldal & Wijkmark, 2017). Therefore, future research should also find out what fields in VET benefit more from VSBT and how to support the creation of VS for different domains. It seems safe to say that VSBT is a good way to train time-critical dynamic decision-making (Polikarpus & Ley, 2021) and problem-solving tasks in different situations.
Limitations and Future Research
We gathered data from a large sample of trainers in the only higher education institute in Estonia training internal security professionals. We used a set of self-report instruments that showed sufficient degrees of reliability and validity. Nevertheless, there are some limitations with the study, specifically (1) the cross-sectional design, (2) the narrow scope of the survey instrument, and (3) the sample size of this study due to the research context.
Because of the cross-sectional research design, our results do not give any evidence of causal relations or temporal dynamics. For example, we do not know whether positive attitudes developed because of the use of VS or it is rather stable. To give more definite answers on how the use and designing of VS differences and helps to integrate the knowledge or change attitudes, a longitudinal research design or pre- and post-collaborative design intervention study should be employed. Trainers should be studied at the beginning of their career and then later when they have gained more teaching experiences or before they are involved in VS creation processes and after.
The survey method allowed us to collect self-assessments from a large number of participants, which necessarily meant that the number of constructs we analysed was limited. Therefore, in the future, this research should be combined with case studies to allow deeper analysis of what kind of VS helps to develop integrated knowledge, as well as to find out about the organisational factors (context) that influence VS adoption in an organisation. To find out how context influences trainers knowledge integration, co-design of virtual simulations should be tested in other countries and organisations, e.g. by applying the Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021). Pre- and post-cross-sectional studies would allow measuring AVSU and TPACK knowledge in relation to their role in using VSBT before introducing VSBT in an organisation and after it is being introduced. It would also enhance the generalisability of VSBT use in VET.
Focusing on just one organisation as a research context had the advantage that context information was taken into account (Antonietti et al., 2022; Cattaneo et al., 2022; Mishra, 2019). On the other hand, this slightly reduced the number of respondents and the power of the statistical tests. Even though the survey was answered by 146 participants and more than half of full-time employed academic staff members in EASS, the size of the VS user groups 4 to 6 (which for us were the most important) was rather small (N = 15 to 17). In order to confirm our findings, the same study could be carried out in other organisations where VS are used to develop vocational competencies, or a cross-organisational study could be targeted. Future research should focus on seeing whether only being a part of creating VS collectively helps trainers to integrate knowledge compared to facilitating VSBT.
Conclusions
This is the first study that looks what is the role of trainers in the implementation process of VSBT in an organisation and how is the role of implementation related to the trainers’ attitude towards the use of VS and TPACK components. Trainers who create VS themselves and train trainers to use VSBT self-assessed their AVSU more positively compared to the trainers who do not use VSBT (H1). Trainers who co-design VS presented higher integrated knowledge compared to trainers who facilitate VSBT. Anyhow, technology knowledge integration to pedagogical and content knowledge should help trainers to implement VSBT in a training organisation context in a specific way. Therefore the approach “Learning by design” using the Collaboration-enriched TPACK framework (Yeh et al., 2021) and The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021) should be further researched in organisations that are planning or already have implemented VSBT.
Availability of Data and Materials
The datasets generated and analysed during the current study are not publicly available due to the fact that the individual privacy of responders could be compromised, and data cannot be anonymised to a sufficient extent to ensure privacy. Also, the data constitutes an excerpt of research in progress. However, the dataset is available from the corresponding author upon reasonable request.
References
Altun, D. (2019). Investigating Pre-Service Early Childhood Education Teachers’ Technological Pedagogical Content Knowledge (TPACK) Competencies Regarding Digital Literacy Skills and Their Technology Attitudes and Usage. Journal of Education and Learning, 8(1). https://doi.org/10.5539/jel.v8n1p249
Antonietti, C., Cattaneo, A., & Amenduni, F. (2022). Can teachers’ digital competence influence technology acceptance in vocational education? Computers in Human Behavior, 132(107266), 1–9. https://doi.org/10.1016/j.chb.2022.107266
Bourdeau, S., Coulon, T., & Petit, M. C. (2021). Simulation-Based Training via a “Readymade” Virtual World Platform: Teaching and Learning with Minecraft Education. IT Professional, 23(2), 33–39. https://doi.org/10.1109/MITP.2021.3062935
Bryant, R. ., Himberg, K. ., Vasile, S. ., Frederiks, M. ., Tikk, M. ., & Kitching, M. . (2019). Institutional Accreditation Report for Estonian Academy of Security Sciences. https://haka.ee/wp-content/uploads/EASS-2019-04.06-final.pdf.
Cattaneo, A. A. P., Antonietti, C., & Rauseo, M. (2022). How digitalised are vocational teachers? Assessing digital competence in vocational education and looking at its underlying factors. Computers & Education, 176, 104358. https://doi.org/10.1016/j.compedu.2021.104358
Deng, F. ., Chai, C. . S. ., So, H. . J. ., Qian, Y. ., & Chen, L. . (2017). Examining the validity of the technological pedagogical content knowledge (TPACK) framework for preservice chemistry teachers. Australasian Journal of Educational Technology, 33(3), 1–14. https://doi.org/10.14742/ajet.3508
Drakos, A., Flandin, S., Filippi, G., Palaci, F., Veyrunes, P., & Poizat, G. (2021). From Exploration to Re-Enactment: Instructional Uses of a Desktop Virtual Environment for Training Nuclear Plant Field Operators. Vocations and Learning, 14(2), 327–352. https://doi.org/10.1007/s12186-020-09261-1
EASS. (2022). Estonian Academy of Security Sciences. Retrived January 31, 2023, from https://www.sisekaitse.ee/en/eass-home .
Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435. https://doi.org/10.1016/j.compedu.2012.02.001
Eutsler, L., & Long, C. S. (2021). Preservice Teachers’ Acceptance of Virtual Reality to Plan Science Instruction. Educational Technology and Society, 24(2), 28–43.
Foronda, C. L., Fernandez-Burgos, M., Nadeau, C., Kelley, C. N., & Henry, M. N. (2020). Virtual Simulation in Nursing Education: A Systematic Review Spanning 1996 to 2018. Simulation in Healthcare, 15(1), 46–54. https://doi.org/10.1097/SIH.0000000000000411
Goss-Sampson, M.A. (2020). Statistical analysis in JASP a guide for Students. https://doi.org/10.6084/m9.figshare.9980744
Harris, J. B., & Hofer, M. J. (2011). Technological Pedagogical Content Knowledge (TPACK) in Action: A Descriptive Study of Secondary Teachers’ Curriculum-Based, Technology-Related Instructional Planning. Journal of Research on Technology in Education, 43(3), 211–229. https://doi.org/10.1080/15391523.2011.10782570
Heldal, I., Wijkmark, C. H., & Pareto, L. (2016). Simulation and Serious Games for Firefighter Training: Challenges for effective Use. Norsk Konferanse for Organisasjoners Bruk Av IT, 24(1), 28–30.
Heldal, I. ., & Wijkmark, C. . H. . (2017). Simulations and Serious Games for Firefighter Training: Users’ Perspective (pp. 868–878). Albi: 14th ISCRAM Conference. Retrived January 31, 2023, from http://idl.iscram.org/files/ilonaheldal/2017/1518_IlonaHeldal+CeciliaHammarWijkmark2017.pdf .
Holtsch, D., Hartig, J., & Shavelson, R. (2019). Do Practical and Academic Preparation Paths Lead to Differential Commercial Teacher “Quality”? Vocations and Learning, 12(1), 23–46. https://doi.org/10.1007/s12186-018-9208-0
JASP Team. (2022). JASP - Free and User-Friendly Statistical Software (Version 0.16.1). https://jasp-stats.org/.
Kardong-Edgren, S. S., Farra, S. L., Alinier, G., & Young, H. M. (2019). A Call to Unify Definitions of Virtual Reality. Clinical Simulation in Nursing, 31, 28–34. https://doi.org/10.1016/j.ecns.2019.02.006
Ke, F., Pachman, M., & Dai, Z. (2020). Investigating educational affordances of virtual reality for simulation-based teaching training with graduate teaching assistants. Journal of Computing in Higher Education, 32(3), 607–627. https://doi.org/10.1007/s12528-020-09249-9
Kemp, A., Palmer, E., & Strelan, P. (2019). A taxonomy of factors affecting attitudes towards educational technologies for use with technology acceptance models. British Journal of Educational Technology, 50(5), 2394–2413. https://doi.org/10.1111/bjet.12833
Keskitalo, T. . (2022). Pedagogical Practices for Organizing Simulation-Based Healthcare Education. International Journal of Learning, Teaching and Educational Research, 21(4), 80–96. https://doi.org/10.26803/ijlter.21.4.1
Kincaid, J. A. (2015). The Relationship between Presence and Performance in Virtual Simulation Training. Open Journal of Modelling and Simulation, 3, 41–48. https://doi.org/10.4236/ojmsi.2015.32005
Koehler, M. .J. ., & Mishra, P. . (2005). Teachers Learning Technology by Design. Journal of Computing in Teacher Education, 21(3), 94–102. Retrived January 31, 2023, http://reference.kfupm.edu.sa/content/t/e/teachers_learning_technology_by_design_84304.pdf .
Koehler, M. .J. ., & Mishra, P. . (2008). Intorducing TPCK. Handbook of technological pedagogical content knowledge for educators (pp. 3–29). Routledge. Retrived January 31, 2023, from https://punyamishra.com/wp-content/uploads/2008/05/koehler_mishra_08.pdf .
Kostov, G., & Wolfartsberger, J. (2022). Designing a Framework for Collaborative Mixed Reality Training. Procedia Computer Science, 200, 896–903. https://doi.org/10.1016/j.procs.2022.01.287
Küttner-Magalhães, R., & Libânio, D. (2022). Virtual Reality Simulation-Based Training: The Way to Go! GE - Portuguese Journal of Gastroenterology, 29(6), 371–373. https://doi.org/10.1159/000527177
Lamb, K., Farrow, M., Olymbios, C., Launder, D., & Greatbatch, I. (2021). Systematic incident command training and organisational competence. International Journal of Emergency Services, 10(2), 222–234. https://doi.org/10.1108/IJES-05-2020-0029
Ley, T., Tammets, K., Sarmiento-Márquez, E. M., Leoste, J., Hallik, M., & Poom-Valickis, K. (2021). Adopting technology in schools: Modelling, measuring and supporting knowledge appropriation. European Journal of Teacher Education, 00(00), 1–24. https://doi.org/10.1080/02619768.2021.1937113
LimeSurvey GmbH. (2021). LimeSurvey - Easy online survey tool. https://www.limesurvey.org/.
López-Bonilla, L. M., & López-Bonilla, J. M. (2011). The role of attitudes in the TAM: A theoretically unnecessary construct? British Journal of Educational Technology, 42(6), E160–E162. https://doi.org/10.1111/j.1467-8535.2011.01232.x
Luik, P., & Taimalu, M. (2021). Predicting the intention to use technology in education among student teachers: A path analysis. Education Sciences, 11(9). https://doi.org/10.3390/educsci11090564
Luik, P. ., Taimalu, M. ., & Laane, H. . (2019). Estonian In-Service Teachers’ and Pre-service Teachers’ Perceptions of Content, Pedagogy, and Technology Knowledge, Based on the TPACK Framework. In t. Väljataga & M. . Laanpere (Eds.), Digital Turn Policy, Practice —Research, in Schools Proceedings of ICEM 2018 ConferenceProceedings of ICEM 2018 Conference. Springer. https://doi.org/10.1007/978-981-13-7361-9_8 (Issue December, pp. 111–122).
Martín-García, A. V., Martínez-Abad, F., & Reyes-González, D. (2019). TAM and stages of adoption of blended learning in higher education by application of data mining techniques. British Journal of Educational Technology, 50(5), 2484–2500. https://doi.org/10.1111/bjet.12831
Mishra, P. (2019). Considering Contextual Knowledge: The TPACK Diagram Gets an Upgrade. Journal of Digital Learning in Teacher Education, 35(2), 76–78. https://doi.org/10.1080/21532974.2019.1588611
Olson, K., Lannan, K., Richards, H., & , Olson, K., Lannan, K., Cumming, J., Macgillivary, H., & Richards, K. (2020). The Concerns-Based Adoption Model and Strategic Plan Evaluation: Multiple Methodologies to Understand Complex Change. Educational Research: Theory and Practice, 31(3), 49–58.
Otto, T. ., & Bauman, H. . (2019). Institutsionaalse akrediteerimise otsus Sisekaitseakadeemia. Estonian Quality Agency for Higher and Vocational Education. Retrived January 31, 2023, from https://haka.ee/wp-content/uploads/SKA-IA-otsus.pdf .
Pamuk, S., Ergun, M., Cakir, R., Yilmaz, H. B., & Ayas, C. (2015). Exploring relationships among TPACK components and development of the TPACK instrument. Education and Information Technologies, 20(2), 241–263. https://doi.org/10.1007/s10639-013-9278-4
Pedram, S. ., Palmisano, S. ., Skarbez, R. ., Perez, P. ., & Farrelly, M. . (2020). Investigating the process of mine rescuers’ safety training with immersive virtual reality: A structural equation modelling approach. Computers & Education, 153, 103891. https://doi.org/10.1016/j.compedu.2020.103891
Põder, S.-F., Savimaa, R., & Link, M. (2015). A framework for training internal security officers to manage joint response events in a virtual learning environment. Proceedings Estonian Academy of Security Sciences: Sustained Security, 151–180. Retrived January 31, 2023, from https://digiriiul.sisekaitse.ee/bitstream/handle/123456789/131/Proceedings 2015.pdf?sequence=1&isAllowed=y
Polikarpus, S. ., & Danilas, K. . (2021). Eesti päästemeeskonna juhtide visiõppepõhise hindamise rakendamine ja tulemused. Turvalisuskompass, 1, 204. Retrived January 31, 2023, from https://digiriiul.sisekaitse.ee/handle/123456789/2844 .
Polikarpus, S. ., & Ley, T. . (2021). Collaborative Authoring of Virtual Simulation Scenarios for Assessing Situational Awareness. In A. . Adrot, R. . Grace, K. . Moore, & Z. . Zobel (Eds.), Proceedings of the 18th ISCRAM Conference (pp. 229–237). Blacksburg Retrived January 31, 2023, from https://www.drrm.fralinlifesci.vt.edu/iscram2021/files/ISCRAM2021Proceedings.pdf .
Polikarpus, S., Ley, T., Hazebroek, H., Edgar, G., Sallis, G., Baker, S., & Masip, A. F. (2022). Authoring virtual simulations to measure situation awareness and understanding. In H. Karray, A. De Nicola, N. Matta, & H. Purohit (Eds.), ISCRAM 2022 Conference Proceedings – 19th International Conference on Information Systems for Crisis Response and Management (pp. 428–433). Retrived January 31, 2023, from https://idl.iscram.org/files/stellapolikarpus/2022/2430_StellaPolikarpus_etal2022.pdf
Polikarpus, S., Ley, T., & Poom-Valickis, K. (2020). Developing the Situational Awareness of Incident Commanders: Evaluating a Training Programme using a Virtual Simulation. Proceedings Estonian Academy of Security Sciences, 19, 195–226. Retrived January 31, 2023, from https://digiriiul.sisekaitse.ee/bitstream/handle/123456789/2595/Proceedings_2020_WEB.PDF?sequence=5&isAllowed=y
Prasolova-Forland, E., Molka-Danielsen, J., Fominykh, M., & Lamb, K. (2017). Active learning modules for multi-professional emergency management training in virtual reality. 2017 IEEE 6th International Conference on Teaching, Assessment, and Learning for Engineering (TALE), 2018-Janua(December), 461–468. https://doi.org/10.1109/TALE.2017.8252380
Reis, V., & Neves, C. (2020). Simulations in virtual reality : Assessment of firefighters ’ decision-making competence. IE Comunicaciones, 31, 28–39.
Rosenberg, J. M., & Koehler, M. J. (2015). Context and technological pedagogical content knowledge (TPACK): A systematic review. Journal of Research on Technology in Education, 47(3), 186–210. https://doi.org/10.1080/15391523.2015.1052663
Sapiano, A. B., Sammut, R., & Trapani, J. (2018). The effectiveness of virtual simulation in improving student nurses’ knowledge and performance during patient deterioration: A pre and post test design. Nurse Education Today, 62, 128–133. https://doi.org/10.1016/j.nedt.2017.12.025
Sarıtaş, T. M. (2015). Chemistry teacher candidates acceptance and opinions about virtual reality technology for molecular geometry. Educational Research and Reviews, 10(20), 2745–2757. https://doi.org/10.5897/ERR2015.2525
Sisekaitseakadeemia. (2018). SISEKAITSEAKADEEMIA ARENGUKAVA 2025 (p. 18). Retrived January 31, 2023, from https://www.sisekaitse.ee/sites/default/files/inline-files/Sisekaitseakadeemia_arengukava_2025.pdf
Schmid, M., Brianza, E., & Petko, D. (2020). Developing a short assessment instrument for Technological Pedagogical Content Knowledge (TPACK.xs) and comparing the factor structure of an integrative and a transformative model. Computers & Education, 157(103967). https://doi.org/10.1016/j.compedu.2020.103967
Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological Pedagogical Content Knowledge (TPACK). Journal of Research on Technology in Education, 42(2), 123–149. https://doi.org/10.1080/15391523.2009.10782544
Sjöberg, D. (2014). Why don’t they catch the baby? A study of a simulation of a critical incident in police education. Journal of Vocational Education & Training, 66(2), 212–231. https://doi.org/10.1080/13636820.2014.896405
Teo, T., Milutinović, V., Zhou, M., & Banković, D. (2017). Traditional vs. innovative uses of computers among mathematics pre-service teachers in Serbia. Interactive Learning Environments, 25(7), 811–827. https://doi.org/10.1080/10494820.2016.1189943
Valk, A., & Kratovitš, M. (2021). We collaborate with everyone, but with some more than others: Evidence of stakeholder collaboration among internal security professional higher education institutions. Empirical Research in Vocational Education and Training, 13(1), 4. https://doi.org/10.1186/s40461-021-00110-6
Wolcott, M. D., & Lobczowski, N. G. (2021). Using cognitive interviews and think-aloud protocols to understand thought processes. Currents in Pharmacy Teaching and Learning, 13(2), 181–188. https://doi.org/10.1016/j.cptl.2020.09.005
Yan, Z. M., Chai, C. S., & So, H. J. (2018). Creating tools for inquiry-based mathematics learning from technological pedagogical content knowledge perspectives: Collaborative design approach. Australasian Journal of Educational Technology, 34(4), 57–71. https://doi.org/10.14742/ajet.3755
Yeh, Y.-F., Chan, K. K. H., & Hsu, Y.-S. (2021). Toward a framework that connects individual TPACK and collective TPACK: A systematic review of TPACK studies investigating teacher collaborative discourse in the learning by design process. Computers & Education, 171(November 2020), 104238. https://doi.org/10.1016/j.compedu.2021.104238
Yerdelen-Damar, S., Boz, Y., & Aydın-Günbatar, S. (2017). Mediated Effects of Technology Competencies and Experiences on Relations among Attitudes Towards Technology Use, Technology Ownership, and Self Efficacy about Technological Pedagogical Content Knowledge. Journal of Science Education and Technology, 26(4), 394–405. https://doi.org/10.1007/s10956-017-9687-z
Acknowledgements
We want to thank all EASS trainers who took the time to fill in the survey. This work was supported by EASS through giving the corresponding author free semester to do the research.
Author information
Authors and Affiliations
Contributions
SP conceptualized the research design, created and piloted the data collection instrument in collaboration with co-authors, collected data and analysed it with help from co-authors and drew conclusions together with co-authors. Also, was main author of the text. KP-V contributed to the creation of the data collection instrument and the reviewing of the article. TL helped to conceptualize the research, advised on creation of the data collection instrument and data analysis and supported the drawing of conclusions. PL reviewed the data collection instrument, helped to conduct data analysis and advised writing methodology and results. The authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing Interests
None.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
A Study of the Use of Technology and Virtual Simulations a | ||
---|---|---|
Studied variable code | Survey items | Question type |
I part b | ||
Pedagogical Knowledge (PK) | Please rate the following statements | Single choice |
pk1 | I can adapt my teaching based upon what students currently understand or do not understand | |
pk2 | I can adapt my teaching style to different learners | |
pk3 | I can use a wide range of teaching approaches in a classroom setting | |
pk4 | I can assess student learning in multiple ways | |
Content Knowledge (CK) | Please rate the following statements in relation to the topic, module or subject that you teach in EASS | |
ck1 | I have sufficient knowledge about my teaching subject | |
ck2 | I can use a subject-specific way of thinking in my teaching subject | |
ck3 | I know the basic theories and concepts of my teaching subject | |
ck4 | I know the history and development of important theories in my teaching subject | |
Technological Knowledge (TK) | Please rate the following statements regarding digital technologies (computers, tablets, mobile phones, Internet, etc.) | |
tk1 | I keep up with important new technologies | |
tk2 | I frequently play around with the technology | |
tk3 | I know about a lot of different technologies | |
tk4 | I have the technical skills I need to use technology | |
Pedagogical Content Knowledge (PCK) | Please rate the following statements with regard to teaching in which you do not use any special technologies or media | |
pck1 | I know how to select effective teaching approaches to guide student thinking and learning in my teaching subject | |
pck2 | I know how to develop appropriate tasks to promote students complex thinking of my teaching subject | |
pck3 | I know how to develop exercises with which students can consolidate their knowledge of my teaching subject | |
pck4 | I know how to evaluate students’ performance in my teaching subject | |
Technological Pedagogical Knowledge (TPK) | Please rate the following statements | |
tpk1 | I can choose technologies that enhance the teaching approaches for a lesson | |
tpk2 | I can choose technologies that enhance students’ learning for a lesson | |
tpk3 | I can adapt the use of the technologies that I am learning about to different teaching activities | |
tpk4 | I am thinking critically about how to use technology in my classroom | |
Technological Content Knowledge (TCK) | Please rate the following statements in relation to the topic, module or subject that you teach in EASS | |
tck1 | I know how technological developments have changed the field of my subject | |
tck2 | I can explain which technologies have been used in research in my field | |
tck3 | I know which new technologies are currently being developed in the field of my subject | |
tck4 | I know how to use technologies to participate in scientific discourse in my field | |
Technological Pedagogical and Content Knowledge (TPCK) | Please rate the following statements in relation to the topic, module or subject that you teach in EASS | |
tpck1 | I can use strategies that combine content, technologies, and teaching approaches that I learned about in my coursework in my classroom | |
tpck2 | I can choose technologies that enhance the content for a lesson | |
tpck3 | I can select technologies to use in my classroom that enhance what I teach, how I teach, and what students learn | |
tpck4 | I can teach lessons that appropriately combine my teaching subject, technologies, and teaching approaches | |
II part | ||
Info | In this study, virtual simulations mean scenarios or problem tasks that are presented using virtual reality (software + hardware) to imitate processes or activities related to working life with the aim of developing the trainees’ competencies. For example, in the context of this study, PyroSim, FireStudio 6 can be considered virtual simulation software, and joysticks, computers, radio transmitters, mobile phones, GPS, etc. can be considered hardware | Information text |
Virtual simulation users´ groups | Which of the following statements best describes your experience with virtual simulations at the EASS? | Multiple choice |
VS user group 1 | I don’t know of any virtual simulations at the EASS | |
VS user group 2 | I know one or more virtual simulation software | |
VS user group 3 | I have tested or used virtual simulations as a trainee | |
VS user group 4 | As a trainer, I use virtual simulations in my training | |
VS user group 5 | I myself create virtual simulations for training | |
VS user group 6 | I advise and give feedback to other instructors regarding the creation of virtual simulations | |
AVSUc | Please rate how much you agree with the following statements | Single choice |
AVSU1 | Nowadays, it is important to use virtual simulations in training | |
AVSU2 | When virtual simulations are used in training, trainees are more motivated | |
AVSU3 | Virtual simulations are a valuable tool in training | |
AVSU4 | Every trainer should use virtual simulations in training | |
III part d | ||
Age | Age | Number input |
Gender | Gender | Single choice |
sugu_M | Male | |
sugu_N | Female | |
Field of teaching | Main field you are teaching at EASS? | Single choice |
eri1 | Languages | |
eri2 | Correction (College of Justice) | |
eri3 | Taxation and customs | |
eri4 | Police | |
eri5 | Rescue | |
eri6 | Social sciences | |
eri7 | Law (trainers part of Centre for Legal and Social Sciences) | |
eri8 | Other (please specify) | |
Pedagogical training | What is your preparation for teaching at the EASS? | Single choice |
SQ001 | I have an occupational qualification certificate as an andragogic, vocational teacher or other similar trainer professional certificate | |
SQ002 | I have participated in the training of teaching methods in the EASS | |
SQ003 | I have participated in the training of teaching methods in some other institution | |
SQ004 | Other (please specify) | |
Job | Who do you work as at the EASS? | Single choice |
ako1 | As a full-time trainer | |
ako2 | As a visiting trainer | |
ako3 | As a manager (e.g. Head of chair, Director of college, etc.) | |
ako4 | As an employee in the maintenance department | |
ako5 | As a support staff member (library, development department, etc.) | |
ako6 | Other (please specify) |
a Introduction of survey in Estonian language was not included in translation. See survey: https://lsurvey.sisekaitse.ee/972213?lang=et) Link is valid until 01.01.2028 AM 12:00.
b All items in I part were single choice items rated with 1 (strongly disagree) to 5 (strongly agree) and translated from Schmid, Brianza, Petko 2020 research. They research a confirmatory factor analysis showed the model to be a good fit (X2(324) = 391.2, p = 0.006; TLI = 0.951; CFI = 0.958; RMSEA = 0.042, 90% CI [0.025; 0.056]; SRMR = 0.071) (Schmid et al., 2020 page 9) This was an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
c Items were adopted from Luik & Taimalu, 2021 questionnaire and word “technology” changed to “virtual simulations” by authors. Internal reliability (Cronbach’s alpha) was 0.848 for attitude toward use in study (Luik & Taimalu, 2021) we adopted the four items to measure AVSU. Our modified items Cronbach´s alpha was a bit higher 0.87 see Table 2 in this article.
d All items in part III were created by the authors of this article.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Polikarpus, S., Luik, P., Poom-Valickis, K. et al. The Role of Trainers in Implementing Virtual Simulation-based Training: Effects on Attitude and TPACK Knowledge. Vocations and Learning 16, 459–486 (2023). https://doi.org/10.1007/s12186-023-09322-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12186-023-09322-1