Introduction

As computer technology grows more powerful, it is now feasible to create computer-based simulations in many domains (e.g. healthcare (Keskitalo, 2022), police (Sjöberg, 2014) and rescue incident commanders’ training (Lamb et al., 2021)). Technologies used to create virtual environments include virtual reality, augmented reality or simply desktop computers or mobile phones that present virtual simulation scenarios (Kostov & Wolfartsberger, 2022). Because of the different technologies used to create virtual simulation scenarios, no unified definition is agreed on cross-disciplinary (Foronda et al., 2020; Kardong-Edgren et al., 2019). In this study, virtual simulations (VS) are defined as scenarios or problem tasks that are presented using virtual reality (software and hardware) to imitate processes or activities related to working life with the aim of developing the trainees’ competencies (Polikarpus & Danilas, 2021). For example, a trainee might encounter a fire scene and needs to decide on a course of action. The actions of he/she in the VS have an influence on the course of the VS scenario, e.g., actions taken (firehose versus fire extinguisher) contribute to stopping the fire (Polikarpus & Ley, 2021). In VET, VS can be used to train dynamic decision-making competencies (Reis & Neves, 2020) or to gain awareness of multi-professional collaboration (Prasolova-Forland et al., 2017). Nonetheless, the integration of virtual simulation-based training (VSBT) into VET is a complex process, and future research should examine efficient design solutions in virtual reality environments (Ke et al., 2020).

Even if VS have been shown to be an effective training tool (Sapiano et al., 2018), most of the current research on VSBT reports the use of VS that has been created for a specific training purpose by experts using standard scenarios. Research studies have been conducted using in-depth expert evaluations (Drakos et al., 2021) or organised as one-off training sessions for specific target groups in a controlled setting or for research purposes (Kincaid, 2015; Sapiano et al., 2018). While such an approach might demonstrate effectiveness in specific settings, it does not give useful guidance on how to create VSBT for specific training contexts in organisational settings in a systematic and sustainable way that would create meaningful learning opportunities for trainees and trainers.

In the organisational context, trainers have a central role in facilitating VSBT (Keskitalo, 2022), as they need to be aware of the capabilities of the technology (Kostov & Wolfartsberger, 2022), need to choose or build these VS for training and eventually integrate these into their training approach. A factor that has limited the wide use of VSBT in training organisations is the need for skilled trainers (Küttner-Magalhães & Libânio, 2022). Especially in VET, research on trainers’ digital competence is still limited, and their attitude towards the use of technology and other contextual factors are important research directions to tackle (Antonietti et al., 2022; Cattaneo et al., 2022).

As there is a scarcity of research on the conditions of the large-scale use of VS in authentic organisational training context focusing on the trainers’ role in the implementation of VSBT, we look at a case of a public Estonian VET provider in which VSBT has been promoted as part of an organisational strategy. The Estonian Academy of Security Sciences (EASS) is the only internal security education training institution in Estonia that provides vocational and continuous training courses (EASS, 2022). It is an institution of professional higher education, characterised by its specialisation to professional qualifications in the fields of police and border guards, rescue, taxation, customs, and corrections at European Qualifications Framework (EQF) levels 4 to 7 (Valk & Kratovitš, 2021). In the EASS, the VSBT has been used in various forms and several named fields for more than twelve years and has therefore developed systematic strategies to integrate it into training practices (Põder et al., 2015; Polikarpus et al., 2020). Due to these circumstances, the case of EASS presents an opportunity to examine the trainers’ role in implementing this type of training.

To implement VSBT in VET, trainers should integrate their technological, pedagogical, and content knowledge with their knowledge of a training institute context (Mishra, 2019). The context in its various levels (micro, meso, macro) should be considered during the studies of trainers’ integrated knowledge (Rosenberg & Koehler, 2015; Schmid et al., 2020). Therefore, in order to implement VSBT into an organisation, the role of trainers needs to be better understood. In this study, we aimed to find out the role of trainers in the implementation process of VSBT in an organisation and how this role is related to the trainers’ attitude towards the use of VS and their Technological Pedagogical Content Knowledge (TPACK). We reported the results of a large-scale survey among trainers at an institution of professional higher education to explore their role in the implementation process of VSBT.

The Role of Trainers in the Use of Virtual Simulations for Training in Vocational and Professional Education

For VSBT to be implemented in an organisation, VS has to be created (Polikarpus & Ley, 2021) or existing ones used (Bourdeau et al., 2021). In both cases, skilled trainers play a key role in effectively implementing VS in training organisations (Keskitalo, 2022). Accordingly, we also focused especially on trainers’ attitudes towards using VSBT and their integrated knowledge of using technologies in training.

Trainers’ Attitude Towards Technology

Practising trainers’ attitudes towards virtual reality technology and the feedback that they provide is an important factor in implementing VSBT (Pedram et al., 2020). As a long history of research into the acceptance of technology shows, a positive attitude towards a new technology has direct effects on the intention to use it (Luik & Taimalu, 2021; Teo et al., 2017), especially in the voluntary context (Kemp et al., 2019; López-Bonilla & López-Bonilla, 2011). The reasons could be that trainers with a positive attitude are readier to experiment with technologies and discover certain uses for VET.

Most studies on the attitude of teachers towards technology use have been conducted in the general education domain (Luik et al., 2019; Teo et al., 2017) and address teacher education students (Luik & Taimalu, 2021; Yerdelen-Damar et al., 2017). The generalisability of VET targeting adult learners needs to be clarified. Moreover, the generalisability of studies that look at the intention to use technology to the actual use of it in VET needs to be better understood. Even more, large-scale studies that could paint a more realistic picture of innovative technology use in VET on macro meaning national or meso meaning organisational level are rare. An exception is a recent studies among Swiss VET trainers (Antonietti et al., 2022; Cattaneo et al., 2022), who found that for practising trainers, attitudes are associated with their knowledge of how to use technology in training (as measured by TPACK).

With regard to the use of VS by trainers in VET, there is even less evidence in the current literature. Some studies exist that look at teacher attitudes towards the use of virtual reality (Pedram et al., 2020) or serious games (Heldal et al., 2016), but these seem to be rather experimental uses of approaches or dealing with implementation challenges. Therefore, we decided to measure working trainers’ attitudes towards VS use in EASS in this paper, referred to as AVSU.

Trainers’ Integrated Knowledge for Using Technology in Training

Studies among pre-service teachers highlight the necessity for theoretical knowledge and practice to be developed for technology integration in education (Altun, 2019). Trainers using VSBT need specific knowledge and skills to use technology. TPACK framework is widely applied to measure teachers’ knowledge related to the integration of technologies into teaching (Mishra, 2019; Yeh et al., 2021). TPACK allows to measure the Pedagogical Knowledge (PK), Content Knowledge (CK), and Technological Knowledge (TK) but focuses especially on the integration of these components Pedagogical Content Knowledge (PCK), Technological Pedagogical Knowledge (TPK), Technological Content Knowledge (TCK), and Technological Pedagogical and Content Knowledge (TPCK) (Koehler & Mishra, 2008; Schmid et al., 2020). In this article, we use the term “non-integrated knowledge” to refer to TK, PK, and CK components and “integrated knowledge” to refer to TPK, TCK, PCK and TPCK components. We use TPACK to talk about all seven dimensions.

For named TPACK components, numerous self-report measurement instruments have been proposed, e.g. by Pamuk et al. (2015) and Schmid et al. (2020). These have been widely applied to measure teachers’ knowledge integration in different domains, such as pre-service chemistry teachers (Deng et al., 2017), mathematic teachers (Yan et al., 2018) or measuring across different domains (Luik et al., 2019). Intriguing research that left out the use of technology in teaching and focused only on economics CK and PCK knowledge found that regardless of trainers’ academic or practical preparation, they had similar CK and PCK (Holtsch et al., 2019). Therefore, we studied all trainers in EASS irrespective of their work experience or employment type.

In VET, however, TPACK is needs to be researched more. Therefore, it is important to find out how personal factors like TPACK knowledge and attitude are related to VET teachers’ digital competencies (Antonietti et al., 2022; Cattaneo et al., 2022). Because the higher the trainers’ integrated knowledge is, compared to non-integrated, the more meaningful and useful knowledge it becomes to make the best use of the possibilities of the technology to support trainees learning (Schmid et al., 2020). As previous studies focus on pre-service teachers’ intended use of new technologies in the context of Estonia (Luik & Taimalu, 2021), it is necessary to look into TPACK knowledge of practising trainers and the relation to patterns of actual use of technology in a training institute – meso level context (Rosenberg & Koehler, 2015) in Estonia. Furthermore, general education in-service teachers’ TK has been found to be of the lowest rank construct followed by TPK (Luik et al., 2019), indicating there might be challenges related to implementing VSBT into training in Estonia.

Participation of Trainers in the Implementation of Virtual Simulation-based Training

Research suggests that contextual factors, such as the amount of organisational leadership or co-workers’ support adopting a technology, influence teachers’ personal adoption decisions (Harris & Hofer, 2011; Martín-García et al., 2019). For adopting technology-enhanced instruction, a factor that seems especially important is to involve teachers actively in the co-design of learning methods and scenarios (Ley et al., 2021). Therefore, it can be assumed that an important dimension of the VSBT implementation in the authentic training organisation context is how much trainers participate in the process.

With regards to development of teachers’ TPACK, “Learning by design” is used, which means they need to map technology and tool(s) affordances to learners’ needs, pedagogical concerns, and specific topics and implement them in a meaningful way into authentic teaching contexts (Yeh et al., 2021). Also, the collaborative design approach to create tools for inquiry-based mathematics learning found that teachers’ PCK plays an important role in the design of pedagogically sound digital tools, and teachers develop their TPK through the implementation of collaborative design of teaching materials (Yan et al., 2018). A collaboration-enriched TPACK framework indicates a theoretical relationship between collective TPACK and personal TPACK in the organisational context (Yeh et al., 2021). Therefore, it could be assumed that through collective processes like co-design, teachers develop collective TPACK, which probably enhances their personal TPACK and vice versa (Yeh et al., 2021).

Here we focus on the role of trainers in implementing VSBT in an organisational context. This is especially relevant for VSBT, as virtual reality software products differ in how much trainers can design the VS scenarios they will use later in training. While some products are shipped as a ready-made training package (with fixed scenarios included), others allow co-design of the scenarios by the trainers. Differences in this type of VS implementation in the concrete context and its relation to trainers’ personal factors, AVSU and TPACK, have not yet been empirically researched.

We conducted our study in EASS to measure the working trainers’ personal TPACK. Because earlier studies in the same organisational context show that trainers can participate in co-designing VS in EASS (e.g. by using The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021)), we assume that this might lead to the development of collective TPACK (Yeh et al., 2021) in EASS. Trainers co-authoring of VS using different theoretical approaches to train situational awareness (Polikarpus & Ley, 2021; Polikarpus et al., 2022) has led to the implementation process of VSBT in EASS (Polikarpus & Danilas, 2021). Therefore, there is a widespread use of VSBT highlighted as an innovative aspect of the EASS approach to teaching (Bryant et al., 2019). In consequence, EASS provides a good testbed for examining the different roles of trainers in the implementation process of VSBT.

Based on research regarding teachers´ acceptance and integration of new technologies into their own teaching (Eutsler & Long, 2021; Martín-García et al., 2019; Olson et al., 2020; Schmidt et al., 2009) and our EASS context knowledge, we indicated six different groups of trainers role in implementing VSBT in EASS in Table 1. The first three groups as variations of non-use and three of use (Olson et al., 2020) VSBT.

Table 1 Virtual simulations user groups in this study
Full size table

Research Problem and Hypotheses

Results of earlier TPACK studies in general education indicate the relationships among teachers’ attitudes towards the use of technology, TPACK, and their plans to carry out technology-integrated lessons (Deng et al., 2017; Luik & Taimalu, 2021). The two most dominant determinants of behavioural intention were TPACK and attitudes towards computer use (Teo et al., 2017). Based on the discussion above, we claim that there is a gap in research on the role of trainers in VSBT implementations in training institutions. Hence, we aim to answer the following research question: What is the role of trainers in the implementation process of VSBT in an organisation and how is the role of implementation related to the trainers’ attitude towards the use of VS and TPACK components?

Given that participation in creating technology-enhanced learning approaches (e.g. "Learning by design") by trainers is an important factor (Yeh et al., 2021), we assume that this will play a crucial role in integrating knowledge, and it will positively influence AVSU. Based on the previous research (Rosenberg & Koehler, 2015) which shows that TPACK is developed in specific context, we believe that the contextual micro, meso and macro level conditions trainers work in when implementing VSBT add to their positive AVSU and integrated knowledge (specially TPK, TCK and TPCK), mutually reinforcing each other. Based on earlier used educational technologies acceptance models (Kemp et al., 2019) we assumed that a meaningful positive attitude contributed to the interest in experimenting with VS. Furthermore, participation of trainers through taking an active role in implementing VSBT (VS user groups 4–6) could eventually contribute to their integrated TPACK knowledge (Yeh et al., 2021). Then using VSBT and especially training trainers to use VS contributes to a further positive attitude towards VSBT. Higher AVSU should help to get started using VS (Sarıtaş, 2015), while participation in the creation of VS helps trainers to integrate their knowledge (Yeh et al., 2021). We illustrate the expected relations between named constructs in Fig. 1.

Fig. 1
figure 1

Trainers’ Role in Implementing VSBT into Organisation Relations to Attitude Towards Virtual Simulations Use and TPACK

Full size image

This has motivated the following hypotheses:

  • H1: Trainers who have created VS (group 5) and who have trained trainers to create VS (group 6) have a more positive AVSU than those who have just used VSBT (group 4) or only tested (group 3) or know about it (group 2).

  • H2: Trainers who have created VS (group 5) and trained trainers to create VS (group 6) have more integrated TPACK knowledge than trainers who have only used VS (group 4) as a ready-made simulation package in their training.

Method

In order to test these hypotheses, a survey study was conducted among 146 trainers of EASS. Respondents had various roles in using VS in their training (independent variable), including active use and creation of VS scenarios. A questionnaire was developed that measured two main dependent variables, trainers AVSU as well as their self-assessment on TPACK.

Sample and Research Context

EASS is the biggest higher-level education state-owned institution under the Estonian Interior Ministry, where current and future staff members of civil servants are trained in four colleges at different loctions in Estonia (Bryant et al., 2019). There are around 1000 trainees’ studying educational curriculums, and in addition, approximately 5000 trainees get continuing yearly training courses in EASS, and they are taught by 85 full-time academic staff members and around 200 part-time trainers (Sisekaitseakadeemia, 2018). It is stated in the international institutional accreditation decision, “The self-improvement, development and evaluation of the academic staff are comprehensive and systematic. They also support the further development of teaching methods, research opportunities and curricula.” and “Teachers are motivated and qualified.” (Otto & Bauman, 2019, p. 5).

The study was conducted among full-time and part-time trainers in EASS. Participation in the survey was voluntary and anonymous. We used the open-source survey software tool LimeSurvey (LimeSurvey GmbH, 2021) and a web-based link was shared through e-mail lists. We downloaded survey participants’ answers from LimeSurvey and excluded the responses from those who had not completed the VS user group question (see Appendix).

Altogether 146 answers were received that could be used for analyses, meaning approximately 51% of all trainers of the organisation responded to the survey. Among survey participants were 50 (34%) full-time trainers, 63 (43%) part-time trainers, 12 (8%) from management and 8 (6%) from support staff with teaching responsibilities, 13 (9%) gave other option answers or did not give their position. The main field of teaching for 46 trainers (32%) was rescue, 38 police and border guard trainers (26%), correction nine trainers (6%), tax and customs seven trainers (5%) and 46 trainers were teaching in different fields or did not answer the question. The average age of participants was 44.9 years (SD 9.1). The youngest trainer was 21, and the oldest was 79. 54% were men (79 male instructors) and 59 females (40%), and eight trainers (6%) preferred not to give their gender.

Concerning trainers’ preparation to teach in EASS, it turned out that 32 trainers out of 146 have an occupational qualification certificate as an andragogic, vocational teacher or other similar trainer professional certificates, 70 have participated in teaching methods training in EASS and 64 in similar training at other institutes. However, 31 participants did not answer the question about their preparation for teaching.

Measures and Instruments

The self-report questionnaire had three parts (see Appendix).

Trainers’ attitude towards VS use – AVSU

For the current study, we adopted items typically used for measuring the construct “attitudes towards technology” in studies using the Technology Acceptance Model (e.g.López-Bonilla & López-Bonilla, 2011; Teo et al., 2017). Specifically, we employed a subscale consisting of four items that had already been translated into the Estonian language and validated by Luik and Taimalu (2021) using a confirmatory factor analysis with standardised factor loadings ranging from 0.690 to 0.826, and scale reliability Cronbach’s alpha of 0.85.

The items were adapted for the current study by replacing “technology” with “Virtual Simulations”. For example, the first item translation into English was “Nowadays it is important to use virtual simulations in training.” All items were rated on a five-point Likert scale (1 “strongly disagree” to 5 “strongly agree”), and all four items are translated into English in Appendix part II.

Trainers’ personal TPACK

We adapted and translated the TPACK-XS instrument, originally developed by Schmid et al. (2020) and validated in a sample of Swiss pre-service upper secondary school teachers (N = 117). A confirmatory factor analysis in that original study had confirmed the factor structure and reliability of the scales. For the present study, all items of TPACK sub-scales were used. They were translated from English into Estonian by the first author and translated back into English by an independent researcher, and the two English versions were then compared and inconsistencies rectified. After that, one Estonian-speaking person made sure terms like “teaching style” and “technology” were always used in the same wording in the Estonian language. The amendments were made only to the introductory text of the original instrument. For example, the phrase “to the subject in which you have written the lesson plan” was replaced with the phrase “module that you teach in the EASS”. All items can be found in the Appendix. The term VS was not used in the first part of the survey.

The translated instrument consisted of the seven TPACK subscales. Each subscale measured with four items in the first part of the survey: 1) Pedagogical Knowledge (PK), 2) Content Knowledge (CK), 3) Technological Knowledge (TK), 4) Pedagogical Content Knowledge (PCK), 5) Technological Pedagogical Knowledge (TPK), 6) Technological Content Knowledge (TCK), and 7) Technological Pedagogical and Content Knowledge (TPCK). All items were rated on a five-point Likert scale (1 “strongly disagree” to 5 “strongly agree”).

VS user Groups

The second part of the survey contained items to establish the independent variable VS user group in line with the grouping introduced in Table 1. First, a definition of VS was provided (see Appendix), followed by the question, “Which of the following statements best describes your experiences with VS so far”. Six statements were used that allowed to creation of an ordinal scale of VS user groups. Mainly to ensure that VS user group questions fit our research context, we conducted several steps to ensure the reliability and validity of our survey instrument (more in chapter Testing Reliability and Validity of the Survey Instrument). First, the survey was piloted, by using a think-aloud protocol (Wolcott & Lobczowski, 2021) with six different experts to make sure the wording of items was clear for VS users and non-users.

Other measures

In the third part, questions were asked about the characteristics of the sample and personal data like respondents’ age, gender, a field of teaching, pedagogical preparation and their role in the organisation. The order of questions in the survey was changed after piloting it with experts, and used VS definition was given in part two prior to the VS user groups question (see Appendix).

Data Analysis

The first dependent variable AVSU was calculated as a mean score from four items (see Appendix). For the second dependent variable (integrated knowledge), we first derived mean scores from each of the seven TPACK subscales (four items each). Then we computed the mean value of the “non-integrated knowledge” from subscales CK, TK, and PK. For “integrated knowledge”, we used the four subscales (PCK, TPK, TCK, TPCK) and computed a measure for “integrated knowledge” as a mean value for each person.

The independent variable (VS user group) was created based on the item “Which of the following best describes your experience with virtual simulations so far?” (Appendix part II). Trainers were grouped based on their “highest order” answers to this question. For example, if trainers answered that “As a trainer, I use virtual simulations in my training” (VS user group 4) and “I myself create virtual simulations for training.” (VS user group 5), then he/she was added to group 5. In the first VS user group (response no 1: “Do not know any VS”) were 34 trainers, and the group were named “No VS info”. The second group (response 2, “Knows one or more VS”) also had 34 trainers and was named “Knows VS”. These were the two biggest groups, so there are 68 trainers (46.6%) who had not used any VSBT as trainers or trainees. The next response was group 3, “Has tested VS” comprised 30 trainers (20.5%) and were also considered non-users of VSBT. In our research, we especially focused on groups four to six. VS user group 4 (15 trainers) named “Has used VS”, group 5 (17 trainers) “Has created VS”, and user group 6 (16 trainers) “Has trained trainers”. All these 48 trainers (32.9%) have implemented VS in their teaching, so they are active VSBT users.

We validated the survey instrument and calculated descriptive statistics for all VS user groups (means, standard deviation, and Skewness and Kurtosis to check for normality). For H1, we conducted a one-way ANOVA (VS user groups as an independent and AVSU as a dependent variable) and a Levene’s test to check the homogeneity of variance and a Dunn’s Post Hoc test. For H2, we conducted a two-way repeated measure ANOVA, where the first within the subject factor was integrated knowledge (integrated vs non-integrated), and the second between the subject factor was VS user groups. We used paired T-test for post hoc analyses. Some of the analyses was done for the whole sample, and some were done only for the groups of trainers actively using VSBT (groups 4–6). These analyses were conducted using Jasp version 0.16.1 (JASP Team, 2022).

Testing Reliability and Validity of the Survey Instrument

We used the responses gathered from our sample (N = 146) to check the reliability and validity of the instrument used: we applied some general measures for it. We calculated Cronbach alpha for each TPACK subscale and AVSU as a measure of internal consistency and used correlation coefficients between survey subscales (see Table 2) to check for the construct validity of the instrument.

Table 2 Pearson’s correlations and cronbach alpha of trainers’ self-assessments for TPACK and Attitude Towards Virtual Simulations Use (AVSU)
Full size table

Table 2 gives an impression of the reliability (internal consistency) by Cronbach alpha, a measure of the intercorrelation of all items in one scale. Given that all Cronbach alpha scores are greater than 0.8, this indicates a highly reliable instrument. In addition, Table 2 presents intercorrelations between the constructs indicating convergent and discriminant validity of the scales as follows: Concerning the intercorrelations between constructs; we expect that these would be significantly lower than the Cronbach alpha scores. As can be seen in Table 2, this assumption is met, as Pearson’s r generally lies between 0.2 and 0.6, with most of them around 0.4. The overall patterns also seem to confirm the idea of TPACK in those correlations of integrated knowledge domains (e.g. PCK) are higher with their respective subparts (e.g. PK and CK) than with the unrelated domain (e.g. TK). Intercorrelations between knowledge (TPACK components) and attitude scale (AVSU) are small to medium for technology-related knowledge (like TPK, TCK, TPCK), confirming prior research on the positive link between technology attitudes and knowledge. For the other TPACK components, correlations with attitude were non-existent.

Table 3 gives further descriptive results for TPACK components and AVSU for all participants of the survey. The trainers have self-assessed the content knowledge (CK) as the highest and the technology knowledge (TK) as the lowest. The standard deviation was highest (SD = 0.83) on items connected to AVSU and lowest on CK (SD = 0.58). We also report in Table 3, PK and integrated PCK, TPK, TCK, TPCK knowledge means and minimum and maximum scores.

Table 3 Descriptive Statistics of the Trainers’ Self-assessed TPACK and Attitude Towards Virtual Simulations Use (AVSU)
Full size table

From correlation studies and descriptive statistics, we conclude that these results provide conclusive evidence of a reliable instrument with sufficient construct validity. We can conclude that the transfer of the instrument to this new context in vocational and professional education was successful.

Results

Trainers’ Attitude Towards Virtual Simulation Use (AVSU)

To illustrate the first hypothesis, trainers who have created VS (VS user group 5) and who have trained trainers to create VS (group 6) have a more positive AVSU than those who have just used VSBT (group 4) or only tested (group 3) or know about it (group 2), Fig. 2 shows means and SD for AVSU for each VS user group.

Generally, AVSU mean values rise for trainers who have used VS in some way (groups 3 to 6) towards more proficient use of VS. It is also seen that those who have trained trainers to build VS have the highest self-assessment for AVSU statements (M = 4.47) and the lowest standard deviation SD = 0.5. Trainers, who have not had any contact with VSBT (groups 1 and 2) had lowest AVSU means (M = 3.68; group 1 SD = 0.81 and group 2 SD = 0.85).

To check whether AVSU varied significantly between VS user groups, a one-way ANOVA was performed. None of the prerequisites was compromised (skewness and kurtosis were in acceptable ranges, and the normality assumption was complied with). The ANOVA resulted in statistically significant differences in self-assessed AVSU between the VS user groups (F = 2.66; p = 0.03; df = 5), although the effect size was small (η2 = 0.09 and ω2 = 0.05). A non-parametric Post Hoc test (Dunn) used for testing small sub-sets of pairs (Goss-Sampson, 2020) showed that only VS user group 6 “Has trained trainers” had statistically significantly higher self-assessments for AVSU than all the other groups (p =  < 0.001).

Fig. 2
figure 2

Trainers’ Attitude Towards Virtual Simulations Use Across Different User Groups

Full size image

Trainers’ TPACK knowledge

To test the second hypothesis, “Trainers who have created VS (VS user group 5) and trained trainers to create VS (group 6) have more integrated TPACK knowledge than trainers who have only used VS (group 4) as a ready-made simulation package in their training.”, we compared each trainer’s “non-integrated” and “integrated knowledge” in their VS user group to which they belonged. Table 4 shows that across all groups, trainers rated their integrated knowledge lower than non-integrated knowledge.

Table 4 Trainers’ self-assessed TPACK knowledge across different virtual simulation user groups
Full size table

As we were especially interested in trainers using VSBT, we restricted the following analysis to only those trainers who were actively using VS in their training (user groups 4–6). In Fig. 3, it is visible that group 4, “Has used VS”, has self-assessed their non-integrated knowledge higher than integrated knowledge, while those differences are smaller for group 5, “Has created VS”, and 6, “Has trained trainers”. To confirm this pattern statistically, we conducted a two-way repeated measure ANOVA with VS user groups 4–6 as a between-subjects factor and integrated knowledge (non-integrated vs integrated) as a within-subject factor.

Fig. 3
figure 3

Trainers’ Self-assessed TPACK Knowledge Across Virtual Simulation User Groups 4–6

Full size image

The ANOVA for VS user groups 4–6 resulted in significant within subjects’ main effects (non-integrated vs integrated knowledge) (F(1) = 7.46; p = 0.01), meaning that across all conditions, trainers were more likely to self-assess their integrated knowledge as lower than the knowledge in the non-integrated components. This is in line with the assumptions of TPACK, where it is assumed that integration of knowledge requires additional explicit learning and knowledge construction processes. The ANOVA also resulted in a significant interaction (integrated knowledge x VS user groups) (F(2) = 3.31; p = 0.05). However, the between-subject effect (VS user groups) had no statistically significant result (F(2) = 1.25; p = 0,3). The assumption of sphericity was not compromised.

To compare trainers’ integrated and non-integrated knowledge self-assessments paired T-test as a post hoc test was used. Because the authoring of VS needs integrated knowledge, we assumed that trainers who have just used but not created VS themselves have statistically significant differences between their self-assessments of non-integrated and integrated knowledge and trainers who have created VS themselves do not. If we compared user group 4 “Has used VS” non-integrated and integrated self-assessments (t = 3.70; p = 0.002; df = 14; Cohen’s d = 0.96), we found a statistically significant difference. However this is not the case for group 5, “Has created VS” (t = 0.08; p = 0.94; df = 16; Cohen’s d = 0.02), and neither for group 6, “Has trained trainers” to use VS (t = 0.88; p = 0.39; df = 15; Cohen’s d = 0.22).

To conclude, the trainers who had used VSBT judged their integrated knowledge lower than their non-integrated knowledge. This was not the case for those trainers who had participated in the creation of scenarios or those who had instructed others. This is in line with our assumption that involving trainers actively in the creation of VS scenarios improves, especially their self-assessment of integrated knowledge.

There is a further trend visible in Fig. 3, namely that group 6, “Has trained trainers”, to use VS seemed to assess their knowledge lower than group 5, “Has created VS”, which seems counterintuitive. However, as the ANOVA did not confirm this effect of a downward trend statistically, it needs to be left to future research to confirm this trend.

Discussion

In this paper, we researched the implementation of VSBT in VET from three perspectives: 1) trainers’ attitude (measured as AVSU) and 2) technology pedagogy and content knowledge (measured as TPACK components), and 3) a trainers’ role in the implementation process of VSBT (measured as six VS user groups). Earlier research found that the attitude towards technology, among other personal factors, contributes to teachers´ digital competence development (Cattaneo et al., 2022). It is clear that a positive attitude towards technology encourages its adoption (Ertmer et al., 2012), but as far as we know, attitudes towards the use of technology have not been researched in the context of VSBT in VET and with regards to its relation to in-service trainers using VSBT.

Also, several studies about TPACK have been carried out among pre-service teachers (Luik & Taimalu, 2021; Schmid et al., 2020; Teo et al., 2017), but the strength and uniqueness of the current study are that it was carried out among vocational trainers in an institution of professional security education. Instead of intentions to use technology in teaching, we researched training practices using VSBT in its specific context. This allowed us to get a more realistic picture of how AVSU and TPACK are related to the actual use of VSBT in a large sample of VET trainers. However, looking at specific groups of users revealed some interesting dynamics in terms of knowledge and attitude.

While self-assessed TPACK knowledge seemed to be rather high throughout the VS user groups, TK was rated the lowest in our sample (except group 1, where it ranked sixth), confirming prior research among in-service teachers in general education in Estonia (Luik et al., 2019). This suggests that integrating new technologies into teaching can be arduous. The highest-ranked non-integrated knowledge component was CK in all VS user groups except group 6, which ranked highest in their PK. If we compared the order of ranking between VS user groups, we noticed that groups 2–4 had exactly the same ranking even if groups 1–3 were not using VSBT and group 4 trainers used VSBT as facilitators. This trend might indicate that being in the role of facilitator of VSBT might not effectively develop trainers’ integrated knowledge. VSBT user groups 5 and 6 rankings were different from non-users of VSBT and group 4 as well as from Estonian general education in-service teachers (Luik et al., 2019). It could be that differences in ranking indicate that trainers who create VS and train others to use them are more involved in collaborative design (Yan et al., 2018) and therefore develop stronger collective TPACK (Yeh et al., 2021).

This study provided meso-level overview of the operationalisations of TPACK (Rosenberg & Koehler, 2015) in EASS. Group 4 (15 trainers) are trainers who act as facilitators of VSBT, and although the importance of their pedagogical practice is highlighted in earlier research (Keskitalo, 2022), we did not find out any meaningful differences in their personal TPACK compared to trainers who do not use VSBT. At the same time, VS user groups 5 and 6 (altogether 33 trainers) probably have developed collective TPACK by taking part in an iterative technology-mapping process, considering tool affordances, trainees’ needs, and pedagogical matters, taking into account important concepts of specific topics in a meaningful way for realisation in the authentic context (Yeh et al., 2021).

We found out that our hypothesis one (H1) was supported. Trainers who use VS in their teaching (especially VS user groups 4–6) have a more positive attitude towards it compared to the trainers who do not use VS in their work. Moreover, trainers who train trainers (user group 6) in the use of VS have a more positive attitude than those using it to facilitate VSBT (group 4). This confirms results from earlier studies that a positive attitude towards the use of technology is related to the intention to use technology and in training others (Luik & Taimalu, 2021). Organisations that intend to boost the use of VS could therefore identify those trainers who already have a positive attitude towards the use of technology and convince them to help others to create suitable learning scenarios for VET VSBT. This approach could be effective because trainers already know the organisation at the micro and meso level (Rosenberg & Koehler, 2015).

Our hypothesis two (H2) was partially supported, as the patterns of results on TPACK showed some interesting differential effects. Specifically, not all TPACK components showed a similar growth from VS user groups 1 to 6, but it was the integrated knowledge that differentiated especially groups 5 and 6 from the rest. Generally, trainers across the user groups judged their integrated knowledge statistically significantly lower than their non-integrated knowledge (within subjects, the main effects of non-integrated vs integrated knowledge was F(1) = 7.46; p = 0.01). It is in line with the ideas in TPACK that integration of knowledge is an additional step to constructing meaningful technology application knowledge and that high self-assessment scores in technology, content and pedagogical knowledge are not by themselves sufficient to ensure meaningful integration of technology into teaching. Trainers learn technology by VS design. However, “design is not something that can be taught by lectures and demonstrations” (Koehler & Mishra, 2005, p. 98). A key finding (as evidenced by the significant interaction effect) is that this difference between integrated and non-integrated knowledge is not found in trainers who create VS scenarios or those who instruct others to do so (VS user groups 5 and 6). We claim that this is evidence that these two groups have developed more integrated knowledge than those who are just using a ready-made training package in their VSBT (VS user group 4). Furthermore, it can be expected that the higher the trainers’ TPK, TCK and PCK are, as compared to TK, PK, and CK, the more meaningful and useful knowledge integration occurs (Schmid et al., 2020), which also would lead to a more meaningful application of those technologies into training. Moreover, both positive AVSU and the development of integrated knowledge are related to the trainers’ role in using virtual reality technology and the creation of VS scenarios. This finding is similar to an earlier study that the use of computers in teaching TPACK and attitude was the biggest influence (Teo et al., 2017).

We also noted a somewhat unexpected result, as descriptively, the group that has trained trainers to use VS (user group 6) tended to judge their knowledge generally lower than those trainers who create VS (group 5, see Table 4). Probably the group of “train the trainers” were evaluating themselves more critically in self-assessment than those who don’t train others to use innovative technology in training. Also, it appears that their lower ratings are especially marked in knowledge relating to content knowledge (CK). It seems that if a trainer needs to give advice to other instructors from his/her own subject field, they self-assess content knowledge lower than pedagogical knowledge. Similarly, user group 5, “Has created VS”, self-assessed their CK as higher than PK. As the ANOVA could not confirm this trend, these interpretations remain speculative for now and should be verified in a larger sample or with more in-depth qualitative studies in one organisation to keep the context of using VSBT as an unchangeable variable.

Obviously, context and organisational support mechanisms are also important prerequisites for the effective implementation of VSBT. Before the use of The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021), the research in EASS stated that there were not enough trainers to implement VSBT (Põder et al., 2015). Our research showed that 23,3% of trainers in EASS have not heard about the possibility of implementing VSBT in EASS. We also found out that 67% of respondents in the survey were not using VSBT, while 48 (33%) were using it. While we have not focused on context related factors in detail in this research, there is some evidence that organisational context has also had an impact on our results. For example, we found that among VS user groups 4 to 6, there are more trainers from the rescue and police college compared to other colleges, centres, or institutes in EASS. Research about VS and serious games use in different countries’ emergency management organisations found that VSBT is seen as a successful way of training vocational competencies (Heldal & Wijkmark, 2017). Therefore, future research should also find out what fields in VET benefit more from VSBT and how to support the creation of VS for different domains. It seems safe to say that VSBT is a good way to train time-critical dynamic decision-making (Polikarpus & Ley, 2021) and problem-solving tasks in different situations.

Limitations and Future Research

We gathered data from a large sample of trainers in the only higher education institute in Estonia training internal security professionals. We used a set of self-report instruments that showed sufficient degrees of reliability and validity. Nevertheless, there are some limitations with the study, specifically (1) the cross-sectional design, (2) the narrow scope of the survey instrument, and (3) the sample size of this study due to the research context.

Because of the cross-sectional research design, our results do not give any evidence of causal relations or temporal dynamics. For example, we do not know whether positive attitudes developed because of the use of VS or it is rather stable. To give more definite answers on how the use and designing of VS differences and helps to integrate the knowledge or change attitudes, a longitudinal research design or pre- and post-collaborative design intervention study should be employed. Trainers should be studied at the beginning of their career and then later when they have gained more teaching experiences or before they are involved in VS creation processes and after.

The survey method allowed us to collect self-assessments from a large number of participants, which necessarily meant that the number of constructs we analysed was limited. Therefore, in the future, this research should be combined with case studies to allow deeper analysis of what kind of VS helps to develop integrated knowledge, as well as to find out about the organisational factors (context) that influence VS adoption in an organisation. To find out how context influences trainers knowledge integration, co-design of virtual simulations should be tested in other countries and organisations, e.g. by applying the Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021). Pre- and post-cross-sectional studies would allow measuring AVSU and TPACK knowledge in relation to their role in using VSBT before introducing VSBT in an organisation and after it is being introduced. It would also enhance the generalisability of VSBT use in VET.

Focusing on just one organisation as a research context had the advantage that context information was taken into account (Antonietti et al., 2022; Cattaneo et al., 2022; Mishra, 2019). On the other hand, this slightly reduced the number of respondents and the power of the statistical tests. Even though the survey was answered by 146 participants and more than half of full-time employed academic staff members in EASS, the size of the VS user groups 4 to 6 (which for us were the most important) was rather small (N = 15 to 17). In order to confirm our findings, the same study could be carried out in other organisations where VS are used to develop vocational competencies, or a cross-organisational study could be targeted. Future research should focus on seeing whether only being a part of creating VS collectively helps trainers to integrate knowledge compared to facilitating VSBT.

Conclusions

This is the first study that looks what is the role of trainers in the implementation process of VSBT in an organisation and how is the role of implementation related to the trainers’ attitude towards the use of VS and TPACK components. Trainers who create VS themselves and train trainers to use VSBT self-assessed their AVSU more positively compared to the trainers who do not use VSBT (H1). Trainers who co-design VS presented higher integrated knowledge compared to trainers who facilitate VSBT. Anyhow, technology knowledge integration to pedagogical and content knowledge should help trainers to implement VSBT in a training organisation context in a specific way. Therefore the approach “Learning by design” using the Collaboration-enriched TPACK framework (Yeh et al., 2021) and The Collaborative Authoring Process Model for Virtual Simulation Scenarios (Polikarpus & Ley, 2021) should be further researched in organisations that are planning or already have implemented VSBT.