Keywords

1 Introduction

Virtual reality (VR) technologies have become a powerful and promising tool in education because of their potential to enrich learning experiences with high immersion and presence experienced by learners [2,3,4]. VR can be described as a mosaic of technologies that support the creation of synthetic, highly interactive three-dimensional (3D) spatial environments that represent real or non-real situations [5]. High-immersive VR environments (iVR) generally involve a head-mounted display (HMD), while low-immersive or desktop VR (DVR) is based on traditional widely accessible devices like mouse and keyboard [6, 7].

Implementing VR in educational settings has been reported highly motivating, increasing student engagement, their presence and supporting the learning process by providing high-quality visualisations [e.g., 8, 9, 6]. Presence is considered as ‘‘the subjective experience of being in one place or environment, even when one is physically situated in another’’ [10]. Immersion, on the other hand, can be classified into physical immersion, a technological attribute that can be assessed objectively [11], and mental immersion, considered as a subjective, individual belief, i.e., a psychological phenomenon [10]. Several meta-analyses have reported positive educational outcomes when using VR, sometimes exceeding those associated with traditional classroom instruction [4, 5, 12,13,14]. The field of higher education has experienced a growing interest in the use of VR technologies showing positive effect sizes in effectiveness of VR interventions in basic science, social science, and engineering [15]. At the same time, using VR technologies in education should be a balanced decision that considers a multitude of factors including its positive and restrictive attributes [3, 16]. The drawbacks associated with utilizing VR technologies are mainly related to the time and costs required for the development of hardware and software [17]. Additionally, there may be potential health and safety concerns with iVR, as well as discomfort associated with wearing head-mounted displays (HMDs). The use of iVR environments may also lead to user distraction and overload, resulting in lower levels of learning comparing to less immersive VR environments [18]. There is also a reported lack in integrating VR into learning scenarios [2, 16].

In most domains, VR is still experimental, and its usage is not systematic or based on best practices [2]. The technologically-heavy development of the software and steep learning curves in the use of VR puts constraints on teachers effectively and flexibly deploying these technologies in their practice and makes educators dependent on solution providers [17]. Therefore CloudClass technology has been developed as a DVR visualisation and presentation tool aiming to overcome the previously identified problems and to foster teachers in creating and customizing the virtual environment that they can use at rather low cost. Having been developed for the broadcasting industry, CloudClass evolved as a low-cost cloud-based solution that can be used in the classroom without big hardware investments and integrated in existing video-conferencing systems. This has potential to enhance the learning experience in the virtual classroom by a possibility to work with virtual spaces (e.g. a room), 3D models (e.g. solar system) and pre-sets of virtual camera positions, importing and blending pre-recorded video material, images and 3D models with streamed video and other digital objects without the need of using HMDs. What is more, CloudClass is being developed as a low-threshold desktop tool that teachers can flexibly use in their practice without having to go through specialized graphic design or programming training, or being dependent on technology developers.

As previous studies identified numerous challenges and barriers for wider institutional implementation of VR tools in educational context [4, 12, 5, 14], this paper aims at identifying the requirements for implementing a DVR tool (CloudClass) for education in the university context to foster efficient, effective and enjoyable learning. Therefore the two research questions are addressed:

  1. 1.

    What are the requirements for implementing and using a DVR tool like CloudClass in university education? (RQ1)

  2. 2.

    What is the feasibility and the importance of the identified implementation requirements for the use of a DVR tool like CloudClass in higher education? (RQ2)

We applied the Group Concept Mapping (GCM) research methodology [19, 20] to collect, objectively aggregate and analyze the opinions of core stakeholder groups – teachers, educational media experts and managers – to depict their shared vision on the potential of implementation of a tool like CloudClass in higher education in the long term.

2 Method

2.1 Group Concept Mapping

Group Concept Mapping (GCM) is a participatory mixed-methods research methodology that identifies a shared understanding of a particular issue or topic that a group of individuals have [19, 20]. It differs from traditional data collection methods by placing the power of idea generation and organization in the hands of the participants, rather than researchers. GCM involves several stages, including brainstorming, sorting, and rating, and uses advanced statistical techniques such as Multidimensional Scaling (MDS) and Hierarchical Cluster Analysis (HCA) to detect patterns and connections within the data, which offer valuable insights into the factors affecting the topic under investigation. Results are presented visually through a concept map, pattern matches and go-zone charts, among others, making them easy to understand and apply [19, 20]. By utilizing the GCM methodology, educational research can gain valuable insights into the factors that impact technology adoption and educational policies.

2.2 Procedure

Employees across two European universities were approached – a distance learning university in the Netherlands – the Open Universiteit (OU) and a traditional Spanish face-to-face-learning university – the University of Santiago de Compostela (USC). The study was reviewed and approved by the Research Ethics Committee of the OU (under the reference number U202200751). Data collection and main statistical analyses were performed in GroupWisdom™ software. Participants were invited to take part in the GCM institution-wide with a post on the intranet, via email, as well as during faculty and department meeting presentations and ongoing events. The information letter outlining the aim of the research and procedure was included in the email or distribution materials together with a link to a 6-min infomercial describing the functionality and affordances of the tool CloudClass. Participants had to give their informed consent in GroupWisdom before starting each stage of the GCM. Anonymous demographic questions about the role in the institution and faculty of affiliation were mandatory to answer for each stage.

Brainstorming.

The purpose of this phase was to generate and collect ideas of the three stakeholder groups across the two universities – OU and USC. Having watched a video introducing the affordances of CloudClass, participants were asked to complete the following focus prompt: “To use/implement CloudClass in education it is required/ needed that…”. For the USC, the GroupWisdom environment was set to Spanish as the interface language, and the focus prompt was translated as: “Para utilizar/implementar CloudClass en la educación se requiere/se necesita…”. Participants were encouraged to add as many ideas as they wished, while keeping each statement limited to one thought and formulating it as if they were completing the focus prompt. Each participant was working independently from others, but could see all the previous contributions under the input field. Participants from the USC were asked to complete the focus prompt in Spanish, while in OU instructions were given in English, but participants could respond in English or Dutch. There was no time limit for completing the brainstorming activity and participants could leave GroupWisdom and return at a later point.

Idea Synthesis.

Upon the completion of the brainstorming phase the generated statements from both universities were combined and edited with the purpose of unifying the format of the data set to be used in the following stages of GCM process. Idea synthesis was an iterative group process that involved researchers from both OU and USC. Because of the different languages, some sentences had to be back-translated from Spanish and Dutch into English for analysis and editing.

While editing, the following criteria were used: each statement represents one idea, so entries having several ideas were split; each statement is unique, so ideas close in meaning were combined without the loss of meaning; each statement is relevant to the focus prompt, so grammatical formulations were adjusted to fit the focus prompt if needed, and ideas out of the scope were weeded out. The value, popularity or priority of ideas was not considered. After the idea synthesis was complete, the unique statements were fed back into the GroupWisdom so that participants could sort them into meaningful groups and rate for feasibility and importance.

Sorting and Rating.

Participants from OU representing three stakeholder groups were contacted again via email, faculty and department meetings and other ongoing events and invited to participate in sorting and rating. During the sorting activity participants were asked to categorize statements into piles according to their view of the meaning and give each pile a name that describes its theme or contents. A statement could be put alone in its own category, and participants were discouraged to create piles according to priority or value. Although there was no time limit for the activity and participants could leave and get back to GroupWisdom if they wanted, it took on average between 30 and 50 min to complete.

After the sorting activity, participants were asked to rate the same set of statements for importance and feasibility on a 5-point scale, from 1 as least important to 5 as most important, and from 1 as least feasible to 5 as most feasible respectively. Participants were encouraged to spread out their ratings using all 5 given options. With higher values in importance scale participants articulated their values, while feasibility displayed the understanding of opportunities and challenges in practical implementation of the CloudClass tool.

2.3 Analysis

The rating activity collected the data that can be analysed with multidimensional scaling and hierarchical cluster analysis of the aggregated coding data.

Multidimensional Scaling (MDS).

Participant grouping of statements was analyzed with MDS to identify the position of each idea relative to others in a 2-dimensional space resulting in a point map. The points that are closer together on a point map indicate a closer relationship in meaning, reflecting the way participants had grouped them together during the sorting activity. This mathematical model was evaluated for accuracy with a routinely-generated stress value index, which for GCM studies typically ranges between 0.20 and 0.35 [21].

The output of the MDS analysis provided information on the anchoring and bridging values of each statement. Statements with low bridging values have been grouped together with statements in close proximity, while those with higher bridging values have been grouped together with some statements further apart from either side. These values are valuable in understanding the point map [19]. By knowing which ideas serve as anchors or bridges, the researchers were able to make more informed judgments about the content. This knowledge also guided their decisions regarding cluster formation before the cluster map was fully completed. Once the cluster array had been decided upon, the average anchoring and bridging values for each cluster were calculated.

Hierarchical Cluster Analysis (HCA).

To organize content into meaningful clusters of statements that aggregate to reflect similar concepts, HCA was applied [22, 23]. Based on the established GCM research and practice [19, 24], the 15-to-4 heuristics was applied, where HCA started with a 15-cluster solution and at each step proposed merging two of the clusters until the 4-cluster solution was reached. A group of four researchers considered the generated cluster solutions individually and discussed the suitability of each cluster solution keeping in mind the purposes of the project.

3 Results

3.1 Participants

Brainstorming. During the brainstorming phase participants of the OU and USC were approached, reaching out to all the university faculties and service media departments of both institutions (see the Annex [1], figures 1–4). Three stakeholder groups were included in this study – teachers/lecturers, educational media experts and university managers (deans of faculties and management board), which was reflected in demographic questions that participants had to complete before each activity. Out of 121 participants who opened GroupWisdom environment at OU, 59 participants across all the six faculties answered the demographic questions and were able to start the brainstorming. In USC out of 50 participants 22 answered the questions, representing 8 out of 28 faculties.

Brainstorming phase was conducted in both universities, and idea synthesis resulted in a single list of final statements. Responses about participants’ role in their university are summarized in Table 1 below.

Table 1. Participant numbers for GCM activities.
Full size table

Sorting and rating phases involved participants from OU across all the faculties. USC participants were not included in the sorting and rating due to organizational factors. The demographic data for each GCM activity collected through participant questions in GroupWisdom is given in Table 2 below.

Table 2. Participation in sorting and rating activities by faculty.
Full size table

3.2 CloudClass Implementation Requirements

As the result of the brainstorming activity 236 ideas were collected (57 at USC and 179 at OU) in response to a focus prompt: To use/implement CloudClass in education it is required/ needed that…. After the idea synthesis this number was reduced to a list of 95 single unique statements that complete the focus prompt (see the Annex [1], Table 1).

Sorting activity and MDS resulted in a graphical representation of a point map (see the Annex, figure 5) with all the 95 statements. The stress value index for this study is 0.27 indicating that the map quite accurately reflects the participants’ original sorting. Bridging values for each statement are also displayed on the point map (figure 5 in the Annex [1]).

Based on the results of the HCA, the group of researchers chose upon the 5-cluster solution as the one reflecting the core themes of the project. The following themes were identified, which are reflected in the cluster names: Evaluation, Institutional Requirements, Maintenance and Training, Student Requirements, Affordances and Infrastructure. These are displayed in the cluster map in Fig. 1 together with the average bridging values for each cluster. The description of each cluster summarizing main themes is given below and the statements within each cluster are presented in the Annex [1].

Fig. 1.
figure 1

5-cluster solution with average bridging values for each cluster.

Full size image

Affordances and Infrastructure.

Within this cluster with the lowest bridging value of 0.26 participants seem to agree on the coherence of this theme, having grouped the statements with the ones close to them on the point map. The theme connects the expectations of the CloudClass system affordances with the requirements for the infrastructure and hardware, both for students and teachers.

Maintenance and Training.

This cluster features two closely related sub-themes. The first embraces teacher training in various formats – from self-study resources like instruction manuals and videos, to in-person training sessions, workshops (e.g. script-writing, using a green screen, lighting), and availability of support by an expert. The second sub-theme reflects the need of awareness of how CloudClass should be used in specific contexts, which pedagogical scenarios are suitable and how they can be translated in technical design.

Evaluation.

This cluster embraces statements around the evaluation of CloudClass in various forms. The array of coherent sub-themes includes the need for structured research and evaluation approaches to explore efficiency and effectiveness of the tool in multiple contexts with different target groups (e.g. learners with disabilities); investigating short- and long-term effects on learning, as well as comparison with other similar tools; involvement of all the stakeholder groups in implementation and evaluation processes; and finally, the expediency (reasonableness) of the tool use in relation to its cost.

Student Requirements.

This cluster covers sub-themes ranging from technical affordances available to students (e.g. manipulating 3D objects, collaboration, usability), their privacy (e.g. data processing, anonymity), agency in shaping the usage of the software (participative design) to the skills’ requirements of students themselves and the effect of using the software on learning (e.g. increase student engagement).

Institutional Requirements.

This cluster has the highest average bridging value of 0.73, meaning least agreement among the participants on the groupings of the statements which were often sorted with statements that were not in their vicinity. Within this cluster there is also a range of sub-themes that participants attribute to the institutional organization. These include the strategic vision of the reason for using CloudClass, accompanied by a clear plan and communication of these widely within the organization. There is a need to define the institutional support in the use of the tool, both in the form of extra working hours, and teaching/academic recognition for educators. Finally, instructional objectives for the use of CloudClass should be clear, transparent, and should be reflected in the recommended instructional design.

3.3 Importance and Feasibility Ratings of the Identified Requirements

Rating data brings another dimension to the analysis making not only numerical, but also visual comparisons possible in the form of pattern matches and go-zones.

Average cluster ratings that were calculated based on individual statement ratings for importance and feasibility were quite high, spanning from 3.69 to 4.17 for importance and from 3.38 to 3.66 for feasibility on a 5-point Likert scale.

The absolute pattern match presented in Fig. 2 illustrates the average rating comparison of importance and feasibility for each cluster with a correlation of r = 0.66 between the two ratings. Here the higher ratings for importance for all the thematic clusters are apparent.

T-tests were produced for each cluster rating. A statistically significant difference was found between the importance and feasibility rating for the following clusters: Evaluation (T-Value = 4.28, p < 0.001) and Maintenance and Training (T-Value = 4.99, p < 0.001); moderately significant difference is seen in clusters Institutional Requirements (T-Value = 2.45, p < 0.05) and Affordances and Infrastructure (T-Value = 2.03, p < 0.05); and no significant difference was identified in the cluster Student Requirements (T-Value = 1.68, p > 0.05). More detail on the T-tests is given in the Annex [1].

Fig. 2.
figure 2

Absolute pattern match for Importance versus. Feasibility.

Full size image

Go-Zones.

To take a closer look at the rating differences within each cluster, go-zones were produced for each cluster. Each go-zone shows the entire content of a particular cluster allocating each statement on the two-dimensional x/y scale according to its rated importance (horizontal axis) and feasibility (vertical axis). All the charts are provided in the digital Annex [1] (in figures 6–10 and tables 2–6), and an example of two go-zones for cluster Evaluation and Maintenance and Training is given in Fig. 3 below.

Fig. 3.
figure 3

Examples of go-zones for clusters Evaluation (left) and Maintenance and Training (right).

Full size image

Based on the means of rating values, go-zone charts divide the scatter plot into quadrants that can be analyzed further. In the context of the CloudClass project, statements located in the upper right quadrant represent immediate or first-priority action points based on participants’ shared vision of highest importance and feasibility. Ideas in the yellow quadrant with high importance rating but lower feasibility (if located at the top of the quadrant) are seen as strategic goals for the project. Finally, statements in the right part of the orange quadrant can be considered as further steps in the workflow of technology implementation, or sub-actions within the major tasks.

4 Discussion

The present study aimed at identifying the requirements for implementing a DVR tool like CloudClass for education in the university context to foster efficient, effective and enjoyable learning. Therefore the two research questions were addressed.

  • RQ1: What are the requirements for implementing and using a DVR tool like CloudClass in university education?

The list of 95 unique individual statements with requirements was produced based on the input from the participants across two universities. The requirements were statistically sorted in five core thematic clusters representing the major requirements that need to be considered for the implementation of CloudClass within an institution: Maintenance and Training, Institutional Requirements, Evaluation, Affordances and Infrastructure, Student Requirements.

  • RQ2: What is the feasibility and the importance of the identified implementation requirements for the use of a DVR tool like CloudClass in higher education?

Feasibility and importance ratings of each individual requirement, as well as thematic cluster were identified and presented in different graphical and statistical ways to inform the implementation process – in the cluster map (Fig. 1), pattern match (Fig. 2), and go-zones (see figures 6–10 and tables 2–6 in the Annex [1]).

Using the GCM methodology, the present study collected and statistically aggregated the views of three stakeholder groups (teachers, educational media experts and managers) within two European universities from different countries – a distance learning university and a face-to-face one, thus depicting the diversity of contexts in the ideas generated within the brainstorming phase. The numbers of participants involved in sorting and rating activities were sufficient to draw robust conclusions for the further analyses.

The requirements for implementation and use of VR technology in education identified in this study go in line with the factors reported in the literature. Financial cost [12, 17], complexity and accessibility of software and hardware [16, 17], the need for evidence-based didactic frameworks [13, 16, 25, 26] and teaching practices that can support teacher training [2, 12], have been mentioned as factors affecting technology adoption as well as effective and widespread implementation of VR tools outside specialized domains like engineering and computer science [2].

This study also took a systematic approach to prioritize the identified implementation requirements according to their importance and feasibility ratings within an institution. The importance cluster ratings on a scale from 1 to 5, ranging from 3.69 (Student Requirements) to 4.17 (Maintenance and Training), and feasibility ratings from 3.38 (Evaluation) to 3.66 (Maintenance and Training) highlighted that all the major factors identified through clustering are of significant importance and high feasibility with a strong correlation between the two. It is worth noting that the order of priority for all the clusters except for Evaluation is the same, emphasizing that all the indicated factors need to be considered early on in the project development. Although it is quite typical to have higher importance ratings in GCM projects as a way for participants to voice their values, the statistically significant difference between importance and feasibility ratings for the clusters Maintenance and Training and Evaluation emphasizes their first-line importance that should be part of technology implementation policy. The former cluster also scored highest on average ratings, bringing about the importance of the identified themes that can be taken as action points in the implementation process. Although cluster Evaluation scored lowest on feasibility, the significantly high importance rating values underscore the urgency to address the research needs early on within the project.

One of the major findings of this study is the result obtained by means of generating go-zones which list the action points for each thematic cluster with a specific level of priority. This way, the present study is addressing an identified absence of systematic approach in the field of VR use for educational purposes [2]. Based on go-zone charts, the statements in the three quadrants of interest were further analyzed qualitatively to inform the CloudClass project. The outcomes of each thematic category are summarized for each cluster in order of average importance and feasibility ratings.

Maintenance and Training.

Statements in this cluster prioritise the ease of meaningful use by teachers and smooth operation for students. Teacher training should include knowledge on why when and how to use CloudClass, in other words, which didactic strategies should be used in specific learning situations. For this purpose instructional materials need to be developed, including hands-on workshops and lectures. In further stages of the project teachers can be supported by previously developed pedagogical scenarios and best practice examples that are readily available. A helpdesk or a CloudClass expert is required to assist teachers with the use of the tool, and after the end of the project continuous maintenance is required.

Institutional Requirements.

Within this cluster the need is voiced to have clear goals for the use of CloudClass that teachers will know about. More specifically, instructional objectives for the use of CloudClass should be defined in specific contexts and supported institutionally by developing teaching practices and suitable instructional design. As a further step, information about CloudClass should be disseminated widely throughout the university.

Evaluation.

Research should be conducted on the effectiveness of CloudClass including short- and long-term effects, comparison with other tools and effects on different target groups (students with disabilities). Further steps suggest forming a working group to evaluate and discuss the added value of CloudClass. It should be mentioned that participants found important that added value of CloudClass would outweigh its costs, but this item scored low on feasibility (2.80 out of 5.00, cluster mean 3.38).

Affordances and Infrastructure.

Within this cluster, interaction support between students and teachers is of primary importance, as well as the possibility of operation with a large group of students. User-friendliness and a short learning curve for using the technology are seen as core features, together with the flexibility of use on the available equipment that students and teachers have, including sufficient connection speed. Apart from that, making CloudClass templates for the use of 3D objects is seen as immediate action point, later including libraries of 3D models available for teachers’ use. Strategic goals include integration of CloudClass in the university eco-system on all levels: hardware and software facilities, as well as appropriate design of learning materials (both instructional and media design).

Student Requirements.

Based on the go-zone analysis of this cluster, participants see the need for CloudClass environment to be easy to use and engaging/appealing as opposed to distracting. Knowing how to balance effectiveness, efficiency and enjoyability within the environment for students is a strategic goal, as well as students being able to opt for a plain background. Technology acceptance and learners’ data protection need to be considered as goals of primary importance.

The present study has some limitations with respect to generalization of the findings. Because participants involved in the sorting and rating phases represented one institution (OU), the results drawn from the analyses can reliably inform the workflow of the specific project set in the Netherlands in a distance-learning university. It would be interesting to compare these findings with those of other institutions, countries and/or developed for other tools. What is more, the majority of participants for the sorting and rating phases were represented by teachers (89% and 83% respectively), whereas it would be interesting to compare this data with other stakeholder groups, potentially including students and representatives of companies developing the tools.

Another limitation derives from the fact that the study was based on evaluating the possible application of DVR tools based on an example of CloudClass. Despite the fact that participants’ responses were triggered by the technical affordances of one tool, we consider the generated output informative and transferable to a range of similar DVR tools that have relatively low technical threshold and do not require extensive training in graphic design or programming.

The results of this GCM study make a solid starting point in addressing technology implementation and integration projects within universities in a holistic and systematic way to ensure effective, efficient and sustainable use and further development of these technologies. Such approach gives voice to the core stakeholder groups and end users and can foster the development of the DVR tool by informing the involved partners of the factors that need to be considered for the successful implementation of similar projects.

5 Conclusions

The present study focused on identifying the requirements for implementing a DVR tool like CloudClass in a university setting. The results of the study provide valuable insights into the priorities and challenges associated with the wide-scale adoption of VR technology in educational institutions.

The findings of this GCM study revealed that the requirements identified across the two European universities (OU, USC) can be grouped into five thematic clusters: Evaluation, Institutional Requirements, Maintenance and Training, Student Requirements, and Affordances and Infrastructure. Go-zones generated from the importance and feasibility ratings for each cluster provide an informative overview of steps to ensure the holistic implementation of a DVR tool like CloudClass. It considers a balance of pedagogical needs and technological affordances of the given tool.

The findings of this study offer insights to the scientific community, addressing the pressing issue of integrating DVR tools into mainstream education beyond experimental endeavours. By shedding light on the perspectives and challenges faced by the core stakeholders, the study not only facilitates a better understanding of their needs but also empowers them to articulate their struggles. Moreover, it strives to bridge the gap between technology developers and educators/practitioners, fostering collaboration, and provides guidance to other similar implementation projects by acknowledging the current experimental phase of DVR tool adoption in educational institutions.

In conclusion, the findings of this study emphasise the importance of considering the identified requirements when implementing DVR tools like CloudClass in educational institutions and offer valuable insights into the priorities and challenges associated with DVR adoption in universities.