The study of virtual reality influence on the process of professional training of miners

Abstract

Virtual reality technologies are actively applied for the organization of professional training in various industries, as well as in distance learning. However, numerous studies show the presence of a large number of negative factors that limit the effectiveness of using these technologies (united by the concept of "cybersickness"). The study, identification and reduction in the influence of these negative factors will increase the immersiveness and quality of the professional training process. Within the framework of this study, several hypotheses have been put forward regarding the negative and positive impact of VR technologies on the process of professional training, the coal and mining industry has been chosen as the subject area. Thus, the problem of effective training of miners for activities in regular and emergency situations is considered, in the latter case, VR technologies would allow forming the necessary set of skills and knowledge about actions in emergency situations. To confirm the declared hypotheses, an experimental group of 30 people was formed, corresponding to the trained miners by age characteristics. Based on the analysis, a list of quantitative and qualitative metrics for evaluating interaction with virtual reality was formed, the software of virtual scenes for two tasks (moving simple objects and a set of exercises in a virtual mine) was developed. The experimental group repeatedly performed these exercises, which allowed us to analyze the dynamics of changes in the average values of quantitative and qualitative metrics. The data obtained were processed by statistical tests (Shapiro–Wilk, Kruskal–Wallis, Mann–Whitney), which allowed us to assess the impact of the selected configurations (with and without VR) and the number of attempts on the selected metrics. The obtained results partially or completely confirmed the declared hypotheses and allowed us to form a list of recommendations for the organization of high-quality professional training using virtual reality technologies.

1 Introduction

1.1 Problem of professional training of miners using virtual reality

The process of professional training of modern specialists is associated with a modeling of a large number of regular and extraordinary situations, performing work duties in conditions of limited time for decision-making and various negative external factors (Sorensen et al. 2013). The readiness of a specialist determines the success of performing the tasks assigned to them, which requires an appropriate level of skills, knowledge, creative thinking and psychological readiness to make optimal decisions in stressful situations (Sommer et al. 2013). It is impossible to form such a level of readiness based only on theoretical approaches; a large amount of practical training is required in conditions close to real ones. In a number of subject areas associated with an increased risk to the health and life of personnel, conducting such training directly in the working environment is difficult and involves large material costs (Pel et al. 2012). A number of emergency situations (accidents, fires, collapses, etc.) are quite difficult to model within the framework of the educational process (Patle et al. 2019).

Therefore, one of the modern and relevant tools of professional training is the use of virtual reality (VR) technologies. The implementation of specialized software that immerses the trainees in a virtual space allows researchers to simulate regular and emergency, develop muscle and visual memory (Liang et al. 2019). This approach is also cheaper and safer in a number of subject areas, in particular it may be the only way to simulate the desired scenario for the teacher (McGrath et al. 2018).

The effectiveness of learning in virtual reality is achieved by blurring the boundaries between theory and practice, game and research content, simulation and working out of real-life situations, specific and accurate working out of problematic moments, the formation of cognitive control (Wenk et al. 2021). However, despite its positive aspects, the use of VR technologies in the process of professional training raises a number of ethical questions related to the use of this technology for trainees with different ages, health status, profession and level of information literacy (Rizzo et al. 2017). Therefore, an important problem is to determine the influence of VR technologies on the quality of professional training and the physical condition of trainees. Within the framework of this study, we are limited to the subject area of coal and mining enterprises and the process of professional training of miners using virtual reality technologies. Despite the known advantages of using these technologies, it is necessary to assess the degree of negative impact of VR technologies, compare VR systems with classical training on a personal computer. The following hypotheses are considered in the course of the research:

H1

The negative influence of virtual reality in some cases can exceed the positive effect of immersion in virtual space, and the negative feelings experienced by a person can interfere with the preparation process and the high-quality performance of tasks.

H2

Virtual reality technologies in the process of professional training expand the capabilities of the trainees in the field of movement, study and interaction with virtual objects, which allows trainees to develop the necessary level of visual and muscle memory.

H3

In the process of interaction with virtual reality, the severity of negative effects may change in the trainee, which is due to the effect of adaptation and habituation.

Thus, hypotheses will be tested. This will allow us to expand the field of knowledge about the negative effects of virtual reality that arise in the process of professional training, to form a list of recommendations and methods for the effective use of VR technologies in the educational process (Bernardes et al. 2015).

The relevance of the considered problem is associated with a high social significance of the task of effective and comprehensive training of miners, the high level of emergency situations in this area, often caused by unforeseen circumstances and natural phenomena, incorrect and erroneous actions of workers (Krasnyansky et al. 2020). Within the framework of this study, VR technologies that immerse the user in the digital world and ensure the immersiveness of the professional training process will be considered. This makes it possible to increase the involvement of the trainee in the educational process, and the complex effect on the main sensory organs increases the influence on the main human modalities (auditory, visual, kinesthetic) (Krasnyanskiy et al. 2018).

The structure of the article includes the following sections. The introduction discusses in detail the problem of the study, the current state of the issue of the use of VR technologies in professional training, the negative effects that arise in this case (in particular, cybersickness), the analysis of existing approaches to the quantitative and qualitative assessment of the process of interaction with virtual reality is carried out. The second section discusses the methodology of the experiment, including a description of the experimental group, the used software, metrics, methods of data collection and carrying the experiment. The third section presents the results of the experimental group passing two virtual scenes, the dynamics of changes in quantitative and qualitative metrics. The following is a discussion of the results, proof of the hypotheses put forward, prospects for further research and the main conclusions.

1.2 Analysis of the prospects of professional training with the use of virtual reality

The development of modern information technologies has led to the emergence of alternative forms of education and the modernization of the education system. The informatization of the training processes is caused by the need to acquire a large amount of useful knowledge in a short period of time. The existing new learning trends enable the learner to perceive the educational material in new forms (storytelling, gamification, virtual reality), have a flexible schedule and be in an informal environment (Dobre et al. 2015; Tikhonova and Raitskaya 2018; Chassignol et al. 2018). During the testing of new technologies in pilot projects, both positive and negative aspects of their influence on both groups of people and on each individual are revealed (Risi and Palmisano 2019). It is possible to assess the feasibility of using these developments on an ongoing basis (Nemtinov et al. 2020) and their implementation in the training system of existing specialists in various fields (Mareev et al. 2018; Belyakova and Saveliev 2019). The research results confirm the prospects of VR technologies in the educational process.

It should be noted that Covid-19 pandemic has made a significant impact on the modernization of the education system around the world. The need to quickly change approaches to the transfer of knowledge and especially the development of practical skills and competencies has exposed a number of serious problems. They have shown themselves most clearly in the field of professional training of specialists in various industries (engineering specialties, workers in chemical, mechanical engineering, transport, mining, etc.), where trainees need a large amount of practical and laboratory classes to develop the required level of competencies (Hu-Au and Lee 2017; Likitweerawong and Palee 2018; Udeozor et al. 2021). Under the conditions of pandemic restrictions, it was not possible to provide the necessary conditions for conducting such a form of training (Goh and Sandars 2020). The use of virtual reality technologies and modeling of real processes in virtual space can solve these problems (Afanasyev et al. 2018). In a number of areas, due to the high cost of conducting experimental research (including the impossibility of modeling them in real production or within an educational institution) or the risk to the health of trainees, these technologies are already successfully used and show high efficiency (Slater and Sanchez-Vives 2016). Thus, training systems based on virtual reality technologies prove their effectiveness on a variety of examples. However, the question of the negative factors that arise in the process of preparation with the use of such systems remains open.

1.3 Existing studies in the area of the negative influence of virtual reality on a person

The use of VR technical and software has certain age restrictions. Oculus VR, Sony (PlayStation VR) companies claim that their VR sets are suitable for users 12–13 years old and older (Tychsen and Foeller 2020), HTC Vive does not specify age restrictions, but it does not recommend using VR headsets for young children. In case of possible consequences for the health and material condition of a person, warnings and agreements are provided that withdraw the responsibility for the damage caused from the manufacturing companies.

Among adults and healthy people (workers of industries, mines, laboratories, etc.) trained using virtual reality technologies, there is a percentage of respondents who note the deterioration of their physical indicators and the impact of negative factors, such as (Yildirim 2020; Porcino et al. 2020; Kim et al. 2021):

• Nausea, dizziness, headache, eye fatigue;

• Loss of orientation, sense of time, reality;

• Distracted attention, loss of focus, fatigue;

• Load on the neck and spine;

• Collision with real-world objects, injury risk.

The consequences of immersion in the digital space may indicate that the user has cybersickness. Cybersickness is an illness that occurs when a person interacts with virtual reality, associated with a conflict between visual information and sensations received by other sensory systems (LaViola et al. 2000). The main symptoms are similar to those of seasickness—nausea, vomiting, dizziness, headache, mental and physical fatigue, drowsiness, impaired concentration, disorientation in space and eye fatigue. Pallor, difficulty swallowing, increased salivation and cold sweat may also appear.

The problem of the influence of virtual reality on human physical performance is noted in numerous studies (Bockelman and Lingum 2017; Dremliuga et al. 2020), but there are still no successful tools for predicting the occurrence of symptoms of this disease. The specific cause of cybersickness is also unknown, but three main hypothetical theories have been proposed (Almeida et al. 2017; Risi and Palmisano 2019).

The theory of sensory conflict assumes that the subject receives contradictory stimulation from the visual, proprioceptive and vestibular human sensory systems involved in virtual reality. Users experience the illusion of self-motion, (Illusions of self-motion (or "vection")), when a person feels a bodily movement despite the fact that no movement occurs. A person may experience illusory movements of the entire body or individual parts of the body, such as arms or legs.

The theory of postural instability describes a violation of the ability to maintain balance in a particular pose or when changing the pose. It is assumed that the negative impact on the physical condition of a person occurs due to the lack of sensory signals that allow determining the correct position of the body in space. And the longer this condition continues, the more serious the sickness will be. For example, a person tries not to fall, deciding on the basis of visual stimuli that he is leaning. Since he is actually motionless, his actions lead to a loss of balance.

The evolutionary theory or the theory of poisons draws a parallel between the symptoms typical for poisoning and the symptoms of cybersickness (in both cases, nausea, problems with body orientation in space, dizziness, etc.). Stimulation of the visual and vestibular systems in virtual reality leads to seasickness, which is picked up by the body as a sign of poisoning.

However, none of the above theories explains why the sickness manifests differently in respondents and has no prognostic power. To determine what factors influence the occurrence of symptoms of cybersickness, it is necessary to understand the relationship between the change in the user's well-being and the used hardware and software. In early studies on this topic, assumptions were made about the dependence of the respondent's age on the manifestations of negative factors (Reason and Brand 1975). In children under 2 years of age, there is usually no deterioration in well-being, and from 2 to 12 years old, the susceptibility of children reaches its maximum. At the age of 12 to 21, the susceptibility to negative factors decreases sharply and after 21 years continues to fall until the age of 50. However, in new studies (Grassini et al. 2021) it is argued that the manifestation of symptoms of cybersickness depends on a much larger number of individual characteristics of a person, although older people in most cases mark their user experience lower than young people, which is associated with perception overload in older subjects (Plechata et al. 2019).

In the studies (Grassini and Laumann 2020; Curry et al. 2020), the dependence of the respondent's gender and the manifestation of negative factors was revealed: women experienced both a more expressed sense of presence and the influence of negative factors. The obtained data were influenced by a number of individual characteristics of the respondents: susceptibility to motion sickness, stress level, individual personality traits (introversion, neuroticism), the level of information literacy, etc.

In the studies (Yildirim et al. 2018; Kourtesis et al. 2019), a detailed analysis of the shortcomings of virtual and augmented reality technologies was carried out. The authors obtained the following results:

Limitations of the hardware:

• Heavy and uncomfortable helmets, large headsets;

• Spatial limitations when moving;

• Unassembled construction (the problem of operational repair);

• High cost of devices, a direct relationship between performance and cost;

• Insufficient display resolution;

• Small viewing angle.

  In the area of software:

• Dependence on the productive power of PC and consoles;

• Insufficient level of graphics quality;

• Lack of direct compatibility with platforms and integration with other programs;

• Poor content optimization, poor performance.

  In the studies (Palmisano et al. 2020), a comprehensive assessment of the problem of cybersickness is also carried out. The authors emphasize that the negative effects are not limited to ordinary nausea and have a specific effect on the human body. In the work, the following main areas of the causes of cybersickness were identified in addition to those listed above:

• Sensory theory of reorganization (significant differences in movement patterns between the real and virtual world);

• The hypothesis of vector conflict (illusion of personal motion);

• The hypothesis of subjective vertical conflict (the difference between the perceived vertical movement and the expected one);

• The hypothesis of the conflict of the remaining frames (mobility / immobility of the elements of the virtual world relative to some reference object, inconsistent with the real world);

• Oculomotor theory of motion sickness (excessive eye muscle tension);

• The hypothesis proposed by the authors about the influence of differences in virtual and physical head posture and the presence of systemic delays in the formation of images in a VR helmet.

Based on the analysis of study in this area, three possible ways to reduce the likelihood of cybersickness were identified: limiting the possibility of movement, active control of the trainee, conducting repeated training. The restriction of the possibility of movement implies that through the restriction of human mobility, the influence of negative factors is reduced; therefore, in many safety recommendations, interaction with virtual reality is offered in one position (sitting in a chair). Active control suggests that postural stability improves if a person independently controls the situation, and is not a passive observer. For example, in virtual driving simulator, drivers are less susceptible to the occurrence of cybersickness than passengers. Multiple repetitions of training with breaks can increase the resistance to motion sickness. It has been shown that repeated 15-min workouts for 3 days give a significant reduction in the subjective and physiological indicators of cybersickness.

Thus, the analysis revealed multiple negative factors that arise when using existing means of polymodal stimulation. There are both the above limitations in the software and hardware of VR equipment, as well as the problems of implementing a virtual scene and integrating a person into it.

1.4 Analysis of existing approaches to assessing interaction with virtual reality

Various qualitative and quantitative indicators can be applied to assess the effectiveness of interaction with virtual reality. By quantitative indicators, we will understand metrics expressed by numerical values, for example, the task execution time, the number of errors, the positioning accuracy of virtual objects (Vosinakis and Koutsabasis 2018). Metrics obtained during medical studies of a person's condition are also of great interest: pulse, EEG, body temperature, etc. (Nam et al. 2018). Qualitative metrics include metrics that reflect the user's subjective impressions of the process of interacting with virtual reality. To collect them, it is necessary to form a list of questions in the form of a questionnaire that allows you to get a comprehensive assessment of the user’s feelings and impressions about the process of interacting with virtual reality. As questionnaires, the results of which can be used to assess the quality and degree of immersion in virtual reality, can be used:

• The questionnaire on the tendency to immerse (ITQ) is used to identify the main personal factors, the presence of which gives a favorable forecast for the emergence of a person's sense of presence while working with a virtual environment. The test evaluates the subjective feelings of the user during their stay in virtual reality and corresponds to the format of the 7-point Richter scale (Weech et al. 2019). The questions relate to the user's tendencies to be involved in the process, the definition of self-perception, concentration, etc.

• The Presence Questionnaire (PQ) is used as a questionnaire that allows you to assess the quality of the interface, its convenience, as well as how it contributes to (or hinders) the user's sense of presence. In this questionnaire, the user is asked to describe his experience of working in a virtual environment by marking the field of a 7-point scale in accordance with the content of the question and descriptive labels (Feick et al. 2020). Questions include the features of the simulated experience, ease of management, compliance of current mechanics with immersion in virtual space, etc.

• The assessment of negative feelings that arise in the user when interacting with virtual reality is carried out using the SSQ questionnaire (Saredakis et al. 2020). The questions are compiled according to the possible symptoms of cybersickness, which are combined into three main factors:

• The nausea factor (Nausea, N) includes symptoms such as a feeling of discomfort, increased salivation, dry mouth, sweating, nausea, difficulty concentrating, abdominal pain and belching;

• Oculomotor activity factor (Oculomotor, O): discomfort, fatigue, headache, eye strain, focusing difficulty, concentrating difficulty, blurred vision;

• Disorientation factor (Disorientation, D): focusing difficulty, nausea, heaviness in the head, blurred vision, dizziness with open and closed eyes, a sense of rotation of the surrounding world.

The user is asked to choose one of the four degrees of his own negative feeling (I do not feel; I feel slightly; I feel moderately; I feel strongly).

• SAN questionnaire is designed to assess well-being, activity and mood (Parfenov et al. 2016). The questionnaire consists of 30 pairs of opposite characteristics, according to which the subject is asked to assess his condition. Each pair is a scale on which the subject notes the degree of actualization of a particular characteristic of his condition. When calculating, the extreme severity of the negative pole of the pair is estimated at one point, and the extreme severity of the positive pole of the pair is estimated at seven points. At the same time, it should be taken into account that the poles of the scales are constantly changing, but positive states always get high points, and negative ones—low. The points received are grouped according to the key (well-being, activity, self-esteem) into three categories and the number of points for each of them is calculated.

• Game Experience Questionnaire (GEQ): a modular tool used to assess the overall user experience of players after completing a game test (Benlamine et al. 2021). It includes three modules. The main module measures seven categories relative to the player's feelings when he plays a video game. The post-game module focuses on what the player feels after playing a video game and measures four aspects: positive and negative experiences, fatigue and return to reality (Johnson et al. 2018). The third module is used to measure social presence.

• The system usability scale (SUS) includes ten points for evaluating the system's compliance with the purpose of its development (Schwind et al. 2019). SUS evaluates three main areas of usability: efficiency (users achieve their goals), effectiveness (the user's efforts and resources are spent on achieving these goals) and satisfaction (user experience).

• ITC-SOPI consists of 44 items that address both various aspects of presence and the negative effects of the virtual environment (Gaggioli et al. 2020). The assessment is made on a five-point scale.

However, the process of professional training is primarily aimed at developing the necessary knowledge, skills and abilities. Therefore, when compiling a questionnaire and forming qualitative metrics for evaluating interaction with VR, it is necessary to take into account the specifics of this subject area.

2 Research methodology

To assess the influence of virtual reality on the process of professional training, it is necessary to conduct experimental studies with subsequent analysis of the obtained data. The main stages of the research methodology will be considered briefly.

At the first, it is necessary to form an experimental group (or several such groups) that will test typical virtual reality systems, including for the selected subject area (within the framework of this study, simulators for training miners). The actions of the experimental group participants are recorded and evaluated according to a number of metrics. Under the assessment of qualitative and quantitative indicators, the numerical values of metrics will be understood. Quantitative indicators are calculated programmatically and reflect the accuracy of the user's interaction with virtual objects, the task execution time and the number of errors made. This software is implemented as a separate code/software in C# and integrates into a virtual scene, tracking user actions in it. The qualitative metrics are formed by filling in a questionnaire. The collection and processing of the questionnaire is an important task within the framework of this study, since it is a comprehensive assessment of the subjective impressions and feelings of the user and should contain both negative factors affecting the quality of interaction and possible changes in the physical parameters of a person performing exercises in virtual reality, as well as have an implemented mechanism for collecting, storing and processing the obtained data.

To fulfill these tasks, it is necessary to develop software for interacting with virtual reality and methods for collecting and processing data. The study ends with the analysis and processing of quantitative and qualitative metrics collected during the passage of virtual scenes, the formation of patterns of VR influence on a person. Next, all the stages of the methodology, the developed software and the proposed methods will be considered in detail.

2.1 Experimental group

The process of forming an experimental group to assess the impact of virtual reality is based on attracting people who meet the following requirements:

• The compliance with the age category of the target audience undergoing training and training to work in the coal and mining industry;

• The gender distribution, close to the real one in this industry (the number of men from 80 to 90%) (Keenan et al. 2016);

• The presence of participants with different experience (including low) in the area of using modern information technologies.

In accordance with these requirements, an experimental group of 30 people was formed, which has the following composition: 25 men, 5 women. The distribution by age of participants is shown in Fig. 1, by the level of information technology proficiency—in Fig. 2.

Fig. 1

Distribution of respondents by age

Fig. 2

Distribution of respondents by level of information technology proficiency

Among the experimental group there are 20 trainees—future specialists in the area of technosphere safety and chemical industry of Tambov State Technical University, as well as 10 employees (engineers, teachers). Thus, by age and gender characteristics, the experimental group corresponds to the target audience undergoing training and training for work in the coal and mining industry. All participants of the group agreed to the processing of their personal data.

2.2 Formalization of quantitative and qualitative metrics

To assess the quality of human interaction with virtual reality in the process of professional training, the quantitative and qualitative metrics based on the experience of other researchers will be formalized (Vosinakis and Koutsabasis 2018).

Each trainee corresponds to his prototype—an information object $u \in U$:

$$ u = (P,M,N), $$

(1)

where $P$ is the set of parameters of the trainee (gender, age, weight, area of activity and others);

$M$ is the set of quantitative metrics;

$N$ is the set of high-quality metrics.

For the determination of quantitative metrics, the following formulas will be used.

The accuracy of interaction with virtual reality objects:

$$ M_{A} = \frac{{\sum\nolimits_{i = 1}^{Q} {\left| {x_{i} - x_{R} } \right|} }}{Q}, $$

(2)

where $Q$ is the number of repetitions of the exercise;

$x_{i}$ is the value of a parameter of some virtual object;

$x_{R}$ is the target parameter values.

In the current study, accuracy $M_{A}$ is the distance between the current $x_{i}$ and optimal $x_{R}$ position of the object, measured in meters.

Time to complete the exercise (measured in seconds):

$$ M_{T} = T_{f} - T_{0} , $$

(3)

where $T_{f}$ is the end time of the exercise;

$T_{0}$ is the start time of the exercise;

The number of errors (normalized from 0 to 1):

$$ M_{E} = \frac{{\left| {X_{L} } \right|}}{Q}, $$

(4)

where $X_{L}$ is the set of virtual objects whose coordinates are outside of the working area of the exercise and their placement is considered erroneous, where $X_{L} = \{ x_{i} |\left| {x_{i} - x_{R} } \right| > \Delta x\}$;

$\Delta x$ is the acceptable error on the selected scale.

It should be noted that the error (4) can be calculated in another way. In this case, the correctness of the sequence of actions is evaluated, not the deviation $\Delta x$ on any scale. In this case, the number of such errors is summed up and divided by the total number of stages.

For each qualitative metric the following formula will be used:

$$ N_{k} = \frac{{\sum\nolimits_{i = 1}^{\left| U \right|} {n_{ik} } }}{\left| U \right|}, $$

(5)

where $\left| U \right|$ is the number of trainees in the sample;

$n_{ik}$ is the answer of i trainee on k problem when fulfilling the exercise.

Based on the analysis of existing metrics (Parfenov et al. 2016; Johnson et al. 2018) and taking into account the need to study the impact of VR on a person, the following list of questions is formulated:

Q1:

Have you experienced nausea?

Q2:

Have you experienced motion sickness?

Q3:

Have you experienced dizziness?

Q4:

Have you been disoriented in space?

Q5:

Is it convenient to interact with virtual objects?

Q6:

Is it convenient to move around the virtual scene?

Q7:

How convenient is the view in virtual space?

Q8:

How convenient is it to recognize the properties of objects (distance, dimensions, details)?

Using a list of given questions, qualitative metrics are collected in two categories—negative effects (Q1-Q4) and convenience of the system (Q5-Q8). Each qualitative metric is calculated according to the formula (5), taking into account the normalization of each question in the range from 0 to 1. For questions Q1-Q4, 0 corresponds to the absence of negative effects, and 1 corresponds to the maximum manifestation of a negative effect. For Q4-Q8, 0 corresponds to a complete lack of convenience, and 1 corresponds to maximum user convenience.

Thus, after calculating by formula (5) for negative effects, it is possible to receive the percentage of those respondents who experienced negative feelings during training, for convenience metrics—an average score. Consequently, to increase the effectiveness of the training process, it is necessary to reduce the number of respondents experiencing negative feelings (Q1-Q4) and increase the average value of the metrics Q5-Q8.

2.3 Structure of software for conducting experimental research

Professional training systems (for example, training complexes) can differ significantly in their functionality, complexity and subject area. To eliminate the influence of the appearance of the scene, the complexity of the exercises performed and other distracting factors, it was decided to use not only a virtual scene with an imitation of a real mine and a simulation of the miner’s activity, but also a simple virtual scene. This is due to the fact that a large set of actions will make it difficult to analyze each individual operation in the virtual space for the user's state.

Therefore, we will use two virtual scenes: a mine model and an empty virtual space with a minimum of objects, so that the user can concentrate on performing the main task of this exercise—moving virtual objects from one point to another. In most virtual simulators for various disciplines and subject areas, the user has to interact with virtual objects and move them, which was the reason for adding this exercise as part of the experiment. Thus, virtual reality will be evaluated within the framework of an empty stage and a simple exercise, as well as when performing a set of exercises in the mine.

To develop virtual scenes, Unity environment was used, Steam VR input system was used to connect HTC Vive VR headset and Open VR was used to correctly support Steam VR. The data collection, processing and storage system is implemented on the basis of Flask framework (Python), MySQL was used as DBMS. The use of Flask made it possible to link the Unity application to the database by using Socket technology.

In the structure of the software, two subsystems can be distinguished—the client (a training complex with information collection functions and a set of virtual scenes) and the server (storage and processing of collected data). The implementation of the database and the storage of information in it is conducted by standard means of the Python language. A closer look at the structure of the client, which includes the following modules will be taken:

• User authorization form: used for user authorization in the client or initial registration, during registration the user enters personal data—full name, gender, age;

• Settings selection form: allows you to select a virtual scene (mine or an exercise for moving objects), control type (mouse/keyboard or VR controllers);

• Modules for collecting quantitative and qualitative metrics: quantitative metrics are collected automatically, qualitative—by filling in a questionnaire after performing exercises;

• A set of virtual scenes.

Each virtual scene includes a set of virtual objects, animations, program code files that implement some scenario, physical process, phenomenon, or their combination. Within the framework of a virtual stage, it is possible to implement exercises that simulate the performance of actions in real life to develop the necessary skills and competencies. In this study, such actions include moving simple objects of various sizes (Fig. 3) and the complex scene “Exercises in the mine” (Fig. 4). Briefly consider the main characteristics of these virtual scenes.

Fig. 3

Task 1 “Moving simple objects”

Fig. 4

Task 2 “Exercises in the mine”

Virtual scene “Moving simple objects.” It is used to practice the skills of moving in virtual space, moving and positioning the simplest objects. The virtual environment is a room with a user’s avatar, a test stand with a set of objects of different sizes. Next to the objects are color-coded receiver areas, in the center of which objects must be placed. The user’s task is to place objects at a given point as quickly and accurately as possible. In one attempt, the user must perform this exercise 10 times.

Quantitative metrics of this scene: accuracy (the average distance between the center of the object and the center of the receiver area, as well as the angle of rotation of the object), time (from the beginning of the exercise to the placement of the last object) and the number of errors (the object is placed outside the required area). It is worth noting what is meant by an error in this exercise. The error occurs if the user “picks up” the object, but does not bring the object to the receiver area and “loses” it. If the object hits the floor instead of the table, then the object is deleted and the error counter is increased by one. At the end of the exercise, the ratio of errors to the total number of exercises is calculated.

Virtual stage “Exercises in the mine.” It is used for practicing a set of exercises in virtual reality: moving, interacting with objects, performing point actions. The virtual environment includes a fragment of the mine space with the user’s avatar, a set of virtual objects (telephone, fire extinguisher, boxes, doors, trolley). The user performs a set of exercises: pick up the phone, put it back, open the drawer, get a fire extinguisher, go to the trolley and put out the fire. After completing the exercise, it is necessary to go further down the shaft, open the door, enter the room and close the door.

Quantitative metrics in this scene are measured as follows: accuracy (the volume of liquid from the fire extinguisher that got into the center of the fire); time (from the beginning of the exercise to the closing of the last door); number of errors (the number of missed stages).

The survey form for collecting qualitative metrics for both virtual scenes is implemented by a separate software class that includes the following functions: output of the questionnaire, processing of button clicks, data storage. Qualitative metrics are normalized and set in the range from 0 to 1 (the user evaluates it on a 5-point scale, which is then normalized). To collect metrics, a questionnaire is issued to the user after the end of each attempt directly in the virtual scene (Fig. 5).

Fig. 5

Interface for collecting quality metrics

After passing the questionnaire, information about the exercise is sent to the database for further processing and analysis.

2.4 Procedure for collecting and processing experimental data

The methodology of data collection and analysis will be considered. It is formalized in the form of a general flowchart (Fig. 6).

Fig. 6

Experimental procedure

The first stage is to authorize the user by registering or identifying the trainee. In the first case, the data about the new user of the system are entered into the system (full name, gender, age, occupation, physical parameters, etc.), in the second case the data are loaded from the database. Thus, each user has a unique identifier.

Next, the user selects the settings of the training scenario—the type of exercise and the control method (for example, in a virtual reality helmet using controllers or on a monitor using a keyboard and mouse).

After the authorization and configuration stage is completed, the modules responsible for monitoring the execution of the training scenario and collecting all the information necessary for further work are launched.

A training scenario represents a set of exercises or actions that the user must perform in virtual reality. An example of such actions may be moving objects, interacting with them, reaching a certain point, or other operations with virtual objects.

Each performed exercise is recorded in terms of several quantitative metrics. The main metrics include the accuracy of the interaction, the task completion time, and the number of made mistakes. The list can be expanded. In parallel, the current coordinates of the user are recorded, which can be used in other subsystems of the training complex, for example, to practice movement. If necessary, it is possible to monitor the physical condition of the user using additional sensors and equipment.

The data collection algorithm works in the background and monitors the parameters of both the trainee and the environment with which they interact.

2.5 Methods of experiments conducting

The methodology of the experiment will be considered. It includes three main stages.

• 1. Entering the personal data of the subject and authorization in the simulator (collection of personal data).

• 2. Passing exercises in virtual reality (collecting quantitative metrics).

• 3. Passing the questionnaire (collecting quality metrics).

At the first stage, each participant of the experimental group enters their personal data. Next, the participant selects a specific configuration of the training scenario. The experiment is carried out on various configurations and repeatedly to obtain data on the growth of the level of skills and abilities of the subjects. As part of the current experimental studies, the following order and combinations of virtual reality components and exercises were selected (Table 1).

Table 1 Set of possible configurations

For convenience the following abbreviation for the configurations are introduced:

MK_MSO. Monitor + keyboard/mouse, the scene “Moving of simple objects.”

VR_MSO. VR helmet + controllers, the scene “Moving of simple objects.”

MK_Mine. Monitor + keyboard/mouse, the scene “Exercises in the mine.”

VR_Mine. VR helmet + controllers, the scene “Exercises in the mine.”

It should be noted why the other combinations of hardware configurations were not used. The "Monitor + controllers" configuration cannot be started without a connected VR helmet, while the camera will be controlled exclusively by the helmet. Tests of this configuration showed that this approach to control with respect to the mouse and keyboard is absolutely inconvenient for users, and the time for completing the exercise increases several times due to the need to manually move the camera. The "VR helmet + keyboard/mouse" configuration is technically feasible, with this approach, the keyboard is responsible for moving and interacting, and the mouse is responsible for controlling a fixed virtual hand. This approach has the following disadvantages: the inability to feel the depth and distance to the object, the lack of natural feedback between the virtual hand and the mouse, a strong motion sickness effect when moving with the keyboard. In cases where it is necessary to integrate the possibility of keyboard input, it is necessary to develop additional software modules, for example, a numeric keypad (Wu et al. 2017; Rajanna and Hansen 2018), or the use of additional controllers (Boletsis and Kongsvik 2019). Thus, both of these approaches cannot be used as a professional training tool due to the identified shortcomings, which justifies their absence in the experimental part.

After completing the exercises at the training complex, a survey should be conducted to obtain a qualitative assessment of the experiment. Taking into account the volume of the experimental group (30 people) and the number of attempts (5 times) for 4 configurations, we get a total number of trials equal to 600. The values of all metrics are averaged (divided by the number of participants in the experimental group), which allows you to reduce the influence of subjective feelings of a particular person and get a more general dynamics of changes in indicators during the passage of exercises. For each test participant, tests on different configurations were conducted on different days to increase the objectivity of the tests.

3 Results

Based on the described test methodology, there will be conducted the experimental studies of the influence of virtual reality on the process of professional training by comparing the values of quantitative and qualitative metrics of configurations based on virtual reality technologies (VR_MSO and VR_Mine) with variants built without their use (monitor, keyboard and mouse)—MK_MSO and MK_Mine.

The scheme of processing and analysis of experimental data has the following form. At the first stage, the data is checked for normality, after which the necessary statistical (parametric or nonparametric) methods of their analysis are selected. Next, the impact of the configuration on the metrics is evaluated. Then, the effect of the number of attempts on the values of metrics is evaluated, that is, the dynamics of the learning process using a posteriori analysis and pairwise comparison of all attempts. As a result of the analysis of the experimental data, the reliability of the hypotheses put forward is verified.

3.1 Statistical analysis of experimental data

We will carry out a statistical analysis of the collected experimental data. At the first stage, we will check the distribution of data for normality for each metric. Among the tests for “normality,” the most common criteria are the normality of Shapiro–Wilk, Kolmogorov, Lilliefors, Anderson–Darling, Kramer–Mises–Smirnov, and others. In this study, the Shapiro–Wilk criteria were chosen for the following reasons: good power characteristics, high efficiency on small sample sizes (up to 50 elements) (Razali and Wah 2011).

The Shapiro–Wilk test was performed using the Shapiro function of the SciPy library, an array of metric values for all configurations and attempts was passed to the input of the function. The algorithm of the Shapiro function calculates a W statistic that tests whether a random sample comes from (specifically) a normal distribution. Small values of W are evidence of departure from normality and percentage points for the W statistic, obtained via Monte Carlo simulation. The Shapiro–Wilk test checks the validity of the null hypothesis: if the null hypothesis is correct, then the data is distributed normally, the alternative hypothesis means that the data does not have a normal distribution. As a result, a value of p = 0.000 was obtained for each metric, which indicates the refutation of the null hypothesis about the normal distribution of the initial data.

For each metric, distribution graphs were constructed with separation for each configuration using the Seaborn library's function histplot. The graphs confirm the conclusions obtained by the Shapiro–Wilk criterion about the absence of normality in distributions. Figure 7 shows the distribution of data by quantitative metrics (accuracy, time and errors). The distribution of data confirms the conclusions made earlier about the longer passage of the scene using VR, on the one hand. On the other hand, VR provides greater accuracy. The number of errors is generally comparable (with the exception of the MK_MSO configuration, where the number of errors is much higher relative to VR_MSO).

Fig. 7

Distribution of data by quantitative metrics. a Accuracy, b Time, c Errors

Figure 8 shows the distribution by qualitative metrics (Q1-Q8). When analyzing the distribution by metrics Q1-Q3, it should be noted that for keyboard/mouse-based configurations (MK_MSO, MK_Mine), respondents' responses were always zero, which leads to a corresponding effect on the first three graphs.

Fig. 8

Distribution of data by qualitative metrics. a Q1 (Nausea), b Q2 (Motion sickness), c Q3 (Dizziness), d Q4 (Disorientation), e Q5 (Interaction), f Q6 (Movement), g Q7 (View), h—Q8 (Recognition)

Analyzing the negative effects on all 4 metrics (Q1-Q4) from the distribution graphs, we can conclude that they are more pronounced for VR-based configurations (VR_MSO, VR_Mine). For interaction convenience metrics, VR-based configurations mostly show the worst indicators, except for the Q8 metric (convenience of recognizing object properties).

Next, we examine the correlation between the metrics and build a heat map (Fig. 9).

Fig. 9

Heat map of correlation between metrics

The presented heat map shows the influence of some features on others: the brighter the color, the stronger the correlation between the variables. The results obtained allow us to draw the following conclusions:

• Configuration K has a strong influence on the accuracy and time of the exercise, as well as on the severity of negative factors (Q1-Q4) and the convenience of recognizing object properties (Q8);

• Attempt P affects the accuracy and convenience of interaction (metrics Q5-Q8), which shows that during training, these metrics significantly change.

Next, we will perform a statistical analysis of the data. In the case of data normality, preference is given to parametric analysis methods (for example, ANOVA) (Grech and Calleja 2018). However, if the data has an abnormal distribution, nonparametric analysis methods should be used, which are the opposite of parametric methods (Awang et al. (2015). Since the Shapiro–Wilk test showed that the collected experimental data are not distributed normally, nonparametric tests will be used for statistical analysis. In the study, we use two main tests.

• 1. Mann–Whitney U-test (Choi et al. 2021). The null hypothesis of this test is that there are no differences between the distributions of the data samples. The rejection of this hypothesis suggests that there is some difference between the samples. This test will be used for pairwise comparison of samples with each other.

• 2. Kruskal–Wallis test (Miclaus et al. 2020). The null hypothesis is that the medians of the totality of all groups are equal. This test is a nonparametric version of ANOVA. It can be used to determine if there are differences between two or more groups. To determine which of the groups is different, a posteriori comparison will be required (for example, using the Mann–Whitney test).

Let's consider the use of these tests for the analysis of quantitative and qualitative metrics. Using these tests will allow us to confirm or refute hypotheses H1-H3 by analyzing the influence of input parameters on output metrics, pairwise comparison of configurations, and studying the dynamics of changes in metrics between attempts (Wenk et al. 2021). This approach has proven its applicability for evaluating experimental data, including in the field of virtual reality technologies (Choi et al. 2021; Miclaus et al. 2020, Wenk et al. 2021). In all tests, we will use the significance level of p Value (p) equal to 0.05.

3.2 The result of the analysis of the dependence of metrics on the configuration

At the first stage, a Kruskal–Wallis test was performed to assess the impact of configurations on output metrics to determine whether there are significant differences between configurations. To do this, let's compare the average values of metrics for each configuration for quantitative (Fig. 10) and qualitative (Fig. 11) metrics. The result of the analysis is presented in Table 2, which presents the average values for each metric for each configuration (Mean) and their standard deviations (SD), as well as the p of the Kruskal–Wallis test. Here and further, the p values below the specified significance level in the tables are highlighted in bold. The Kruskal–Wallis test showed that the null hypothesis for all metrics is refuted, that is, there is a significant difference between the configurations for all metrics, so we will conduct a pairwise comparison of the configurations using the Mann–Whitney U-test (Table 3). We compare configurations in pairs within the same tests, respectively, we get a comparison of MK_MSO and VR_MSO, MK_Mine and VR_Mine. For those pairs where the Mann–Whitney test shows significant differences (p < 0.05), we add brackets to the graphs (Figs.10 and 11).

Fig. 10

Average estimates of quantitative metrics obtained for different configurations. a Accuracy, b Time, c Errors. *—p < 0.05 in the Mann–Whitney test. Error bars: ± 1 SD

Fig. 11

Average estimates of quality metrics obtained for different configurations. a Q1 (Nausea), b Q2 (Motion sickness), c Q3 (Dizziness), d Q4 (Disorientation), e Q5 (Interaction), f Q6 (Movement), g Q7 (View), h Q8 (Recognition). *—p < 0.05 in the Mann–Whitney test. Error bars: ± 1 SD

Table 2 Results of the Kruskal–Wallis test (evaluation of the impact of configurations on metrics)

Table 3 Mann–Whitney test results (configuration comparison)

To confirm hypotheses H1 (about the greater severity of negative factors in VR) and H2 (about the effectiveness of VR in the field of professional training), we will conduct a pairwise comparison of all configurations using the Mann–Whitney test.

For the metric of the number of errors in Table 3, the null hypothesis of the Mann–Whitney test is not rejected, that is, the error distributions in MK_Mine and VR_Mine are equal. This means that in the “Exercises in the mine” scene, the number of errors does not depend on the type of equipment used (VR or keyboard/mouse). Indeed, for this scene, an error indicates an unfulfilled exercise and is often caused by the fact that the user forgot to go through a certain stage or do the right action. In contrast, in the “Moving of simple objects” scene, errors directly depend on the convenience of interacting with the environment and the equipment used.

For the other metrics the null hypothesis is refuted, the distributions of these metrics are not equal, which allows them to be compared. Analyzing the average values (Table 2), we get that the VR-based configuration shows greater accuracy and fewer errors in the “Moving of simple objects” scene, which corresponds to the H2 hypothesis. In addition, the VR-based configuration in both scenes according to the Q8 metric (convenience of recognizing the properties of objects) shows the best results, which also has a positive effect on professional training (in the field of visual memory about the properties of objects and the environment). In the “Exercises in the mine” scene, on the contrary, the keyboard and mouse allow you to perform actions more accurately. In both scenes, VR configurations show a longer time to complete the exercise. Evaluating qualitative metrics, we can conclude that there is a significant increase in the severity of negative effects (metrics Q1–Q4) on configurations with VR, which confirms the hypothesis H1. Thus, both hypotheses are confirmed.

3.3 The result of the analysis of the dependence of metrics on the number of attempts

Next, we will use the Kruskal–Wallis test to confirm the H3 hypothesis (reduction in negative VR factors in the preparation process). It is necessary to test the null hypothesis that the medians of the totality of all groups for each attempt are equal for individual configurations. If the null hypothesis is rejected, there is a difference between the groups, which will confirm hypothesis H3.

The results of the Kruskal–Wallis test for each configuration are presented in Table 4. For the MK_MSO configuration from Table 4, it can be concluded that with each attempt, users improve the passage time of the first scene and reduce the number of errors, and the effect of disorientation in space also decreases. Other metrics cannot be evaluated (p > 0.05). When using the keyboard and mouse (MK_Mine configuration), the difference between attempts is confirmed by the metrics of time, number of errors, Q4 and Q6. This is due to memorizing the sequence of actions in the mine, which leads to improved performance.

Table 4 Kruskal–Wallis test results (evaluation of the impact of the number of attempts on metrics)

For the VR_MSO configuration, the difference in almost all metrics is confirmed, which confirms the improvement as quantitative (accuracy, time, number of errors) so are quality metrics. The greatest importance in the framework of the study is the decrease in the values of negative metrics (Q1–Q4) with each attempt. The same results were obtained for the VR_Mine configuration as for VR_MSO: with each attempt, the influence of negative factors decreases, and the remaining indicators improve. In the scene “Exercises in the mine,” due to the specifics of the exercise (fire extinguishing), the hypothesis of equality of distributions according to the accuracy metric was not confirmed.

Next, we will perform a posteriori analysis, as well as visualize the dynamics of changes in quantitative and qualitative metrics with an increase in the number of attempts. Since, within the framework of the H3 hypothesis, we are interested in the dynamics of changes in metrics using VR-based configurations, then further on the graphs we would mark with brackets significant differences in the Mann–Whitney test (p < 0.05) only for VR_MSO and VR_Mine.

Figure 12a–c shows the results of processing quantitative indicators for the scene “Moving of simple objects” (configurations MK_MSO and VR_MSO), Fig. 12d–ffor the scene “Exercises in the mine” (configurations MK_Mine and VR_Mine). The histograms correspond to the average values of the metrics for the group within each attempt. Error bars correspond to ± 1SD (the standard deviation of the metric within each attempt).

Fig. 12

Average estimates of qualitative metrics obtained for different attempts. a Accuracy in the scene «Moving of simple objects», b Time in the scene «Moving of simple objects», c Errors in the scene «Moving of simple objects», d Accuracy in the scene «Exercises in the mine», e Time in the scene «Exercises in the mine», f Errors in the scene «Exercises in the mine». *—p < 0.05 in the Mann–Whitney test. Error bars: ± 1 SD

In the first scene (Fig. 12a), the analysis of the results shows that in the experimental group, after a complete passage (5 attempts) of the exercise, greater accuracy in performing actions was obtained on the VR_MSO configuration based on the helmet and VR controllers. This is due to the fact that the user can more accurately position the controller in space, estimate the distance to the object, and the image transmitted to the helmet corresponds to the viewing angle in the real world with a high degree of accuracy. The selected list of factors allows users to quickly master this method of interaction with the virtual space and get high performance. Using the user’s keyboard and mouse during the first pass is easier to perform the exercise; however, with the number of attempts, the accuracy does not increase as much as when using VR equipment. Having got used to VR controllers, users quickly compensate for the initial lag in accuracy, after which they get better results with it, which, however, differ in a large spread caused by the individual degree of adaptation and learning ability of a person.

In the second scene, the accuracy is higher on MK_Mine configuration (Fig. 12d), since it is determined by the ability to correctly position the fire extinguisher when extinguishing a fire, which for most users corresponds to the already existing gaming experience using the keyboard and mouse.

The task execution time when moving simple objects using the keyboard and mouse (configuration MK_MSO) is less (Fig. 12b), since this approach allows you to move around the scene and place objects in a way familiar to most users. In the mine (Fig. 12e), due to the greater complexity of actions (opening drawers and doors, taking fire extinguishers, etc.), this effect is slightly reduced for MK_Mine. However, a large number of actions that require the correct movement of hands with controllers lead to an increase in the total time of the exercise on the VR configuration (VR_Mine). It is worth noting that for a number of users, both methods are comparable. These group members show similar results in terms of the time of passing the exercises, having got used to control by means of VR controllers.

The number of errors in the first exercise (Fig. 12c) is higher for the keyboard-mouse combination: despite the usual control scheme, users often placed objects past the receiver due to incorrect distance estimation, which caused them to fall more often, and the attempt was counted as erroneous. In the second exercise (Fig. 12f), the number of errors is comparable, since the correctness of the sequence of actions was evaluated, the control method did not have much influence on this parameter.

The Mann–Whitney test showed that there is a significant difference between attempts for all metrics (except for the accuracy for the "Exercises in the mine" scene, since the results of the Kruskal–Wallis test do not allow us to evaluate this metric). It should be noted that a clear, significant difference does not always manifest itself between neighboring attempts. Sometimes the major difference appears only between the first and fourth or the first and fifth attempts. According to the authors, this is due to the user's addiction to VR, when the positive effect does not appear immediately, but after some time.

Next, let's look at qualitative metrics and the dynamics of their changes. Thus, we will get a set of histograms reflecting the process of changing qualitative metrics as we go through the exercise on each configuration. The ordinate axis will denote the dimensionless value of the arithmetic mean of the group members’ estimates for each question. Error bars correspond to ± 1SD (the standard deviation of the estimates within each attempt).

First, a group of 4 questions (Q1-Q4) will be considered, which correspond to negative factors affecting a person: nausea, motion sickness, dizziness and disorientation. Figure 13 shows the dynamics of changes in quality indicators for all configurations at once, the upper graphs correspond to the scene “Moving of simple objects” (MK_MSO and VR_MSO), the lower ones correspond to “Exercises in the mine” (MK_Mine, VR_Mine).

Fig. 13

Values of qualitative metrics Q1-Q4 for each attempt. a Q1(Nausea) in the scene «Moving of simple objects», b Q2 (Motion sickness) in the scene «Moving of simple objects», c Q3 (Dizziness) in the scene «Moving of simple objects», d Q4 (Disorientation) in the scene «Moving of simple objects», e Q1(Nausea) in the scene «Exercises in the mine», f Q2 (Motion sickness) in the scene «Exercises in the mine», g Q3 (Dizziness) in the scene «Exercises in the mine», h Q4 (Disorientation) in the scene «Exercises in the mine». *—p < 0.05 in the Mann–Whitney test. Error bars: ± 1 SD

During the test, there was a significant difference in the negative feeling of nausea (question Q1) in the case of VR equipment compared to the perception of the scene from the monitor. This is due to the following factors—the lack of experience of interacting with VR equipment (proved by a decrease in the average level of negative influence with an increase in the number of passes of the exercise and, consequently, adaptation to a virtual reality helmet). There is some difference between the first exercise (moving objects, Fig. 13a) and the second (“Exercises in the mine,” Fig. 12e), since in the first situation the user moves around the virtual space for an insignificant distance, and in the second the number and complexity of such movements increases significantly, which leads to an increase in the nausea effect and the preservation of its level as a whole for the group in a noticeable size. For configurations MK_MSO and MK_Mine, the severity of this effect is due to the peculiarities of the organism of some participants; in general, this effect can be considered absent for the group.

The motion sickness effect (question Q2, Fig. 13b and f ) occurs when working with VR equipment and actively moving in a virtual reality helmet. The survey showed that this negative effect is generally much more common than nausea. On the monitor, users practically do not encounter such an effect.

The effect of dizziness (question Q3, Fig. 13c and g) is shown when using configurations VR_MSO and VR_Mine and active movement. Vertigo when using VR occurs with comparable frequency in both exercises (VR_MSO, VR_Mine) and manifests itself in many users. When using the mouse and keyboard (MK_MSO, MK_Mine), users practically did not experience this effect.

The disorientation effect (question Q4) manifests itself to varying degrees for all configurations, as it is associated with the ability of users to position themselves in virtual space. When using VR equipment, due to the novelty of sensations and lack of experience, the effect is also more pronounced compared to the monitor. This effect quickly fades due to the rapid adaptation to the features of the review in virtual reality (Fig. 13d and h). In a simple scene for MK_MSO, the effect is weakly expressed due to the small need for movement and orientation in space. In the case of a complex virtual scene (“Exercises in the mine,” Fig. 13h), the disorientation effect is more pronounced for any type of exercise, since it takes a lot of time for the respondent to get used to the more complex structure of the virtual space.

Thus, in the course of experimental studies for virtual reality-based configurations, the negative effects (nausea, motion sickness, dizziness and disorientation) were revealed in some subjects. During the exercises, some of them adapted to virtual reality. The most widespread effect is dizziness, disorientation due to lack of experience with virtual reality, the need to adapt to an unusual viewing angle.

Next, the factors of convenience of interaction with virtual reality (Fig. 14) will be considered: the upper graphs correspond to the scene "Moving simple objects" (MK_MSO and VR_MSO), the lower ones correspond to “Exercises in the mine” (MK_Mine, VR_Mine). Error bars correspond to ± 1SD (the standard deviation of the estimates within each attempt). The obtained results reflect the positive dynamics of changing metrics: with each attempt, it becomes more convenient for users to interact with objects and move around in virtual reality.

Fig. 14

Values of qualitative metrics Q5-Q8 for each attempt. a Q5 (Interaction) in the scene «Moving of simple objects», b Q6 (Movement) in the scene «Moving of simple objects», c Q7 (View) in the scene «Moving of simple objects», d Q8 (Recognition) in the scene «Moving of simple objects», e Q5 (Interaction) in the scene «Exercises in the mine», f Q6 (Movement) in the scene «Exercises in the mine», g Q7 (View) in the scene «Exercises in the mine», h Q8 (Recognition) in the scene «Exercises in the mine». *—p < 0.05 in the Mann–Whitney test. Error bars: ± 1 SD

Since most of the experimental group represents an age group ready to study new technologies with a sufficiently flexible mindset and a high degree of adaptability, despite having more experience with the keyboard and mouse, the group quickly adapts to the new method of interaction based on VR. Moreover, subjective user ratings show that the novelty of this method and the positive impressions received from interacting with virtual reality lead to the group members initially giving higher ratings than could be expected. This specificity of the assessment of the training process can indirectly positively affect the motivation of trainees, which is also an additional advantage of using VR.

According to the metric of convenience of interaction with virtual objects (question Q5), initially it is more convenient for users to use the keyboard and mouse (configurations MK_MSO and MK_Mine), but by the end of the experiment, the difference between configurations is leveled (Fig. 14a and e).

The assessment of the quality of the convenience of movement (question Q6) depends very much on the number of movements performed by the user. The user in the scene “Exercises in the mine” performs a large number of movements, which leads to the negative effects of motion sickness and dizziness discussed earlier (Fig. 14f). This leads to lower scores on the first attempts for configurations VR_MSO and VR_Mine. However, after getting used to moving using controllers, users at the end of the experiment evaluate all configurations in general in the same way (Fig. 14b and f).

The situation is similar for the convenience metric of the review (question Q7). The need to adapt to the specifics of the virtual reality helmet initially leads to lower estimates for the VR_MSO and VR_Mine configurations, which increase with increasing participant experience (Fig. 14c and g). The overview with the mouse is familiar to most users and the amount of change in this metric for configurations MK_MSO and MK_Mine is insignificant.

According to the metric of convenience of recognizing the properties of objects (question Q8), VR-based configurations during the entire experiment show an advantage. It is due to the effect of a more realistic representation of three-dimensional reality. Seeing his or her own hand with the controller, it is much easier for the user to correlate the distance to objects and between them, their sizes. In addition, in a virtual reality helmet, scaling of the virtual space is the most natural and closest to the real one. It should be noted that in the "Moving of simple objects" scene (Fig. 14d) we cannot reliably estimate the Q8 metric, this may be caused by the simple structure of objects (cubes). On the other hand, in the "Exercises in the mine" scene (Fig. 14h), statistical analysis confirms the improvement of the Q8 metric between the first and fourth/fifth attempts.

Thus, the H3 hypothesis is also confirmed, since for all negative factors (Q1-Q4), an improvement in the values of metrics was noted (Fig. 13). In addition, this indirectly confirms the improvement of quantitative metrics (Fig. 12) and user ratings on interaction convenience metrics (Fig. 14).

4 Discussion

The studies conducted on an experimental group of 30 people allow us to draw the following conclusions on the impact of virtual reality on the process of professional training. The use of virtual reality requires additional training to develop the necessary skills to work with this technology. During the experiments, it was noted that even 5 attempts to perform the exercise lead to a significant improvement in the accuracy of interaction with VR, a reduction in the time of the exercise and the number of made mistakes. Continued training can improve these indicators. An untrained user at the beginning of training shows lower results not because of the level of their professional skills, but because of an insufficient level of knowledge of VR technologies.

On configurations with a keyboard and mouse, the exercise time is lower than using a virtual reality helmet, which is due to the participants’ extensive experience in working with this type of control. When using virtual reality controllers, users make fewer mistakes when positioning objects due to a more reliable evaluation of the distances and scales of objects in virtual space.

Training systems technologies, based on virtual technology, have a fairly common negative effect on an unprepared user. Even after repeated repetition of the exercise, users experience negative effects: up to 19% of cases they experience nausea, in 3–13%—motion sickness, in 5–12%—dizziness, in 10–14%—disorientation. During the first run of the exercise, the negative effects are even more expressed. The participants who moved relatively little during the exercise noted a weak severity of negative effects. Thus, in the case of working at virtual stands, tables, equipment without the need for direct movement, we can talk about a fairly low level of negative effects. If the work is connected with the active movement of the user, then it is necessary to carry out careful preparation of the user for possible negative consequences.

The participants noted the high convenience of performing actions with virtual objects using VR controllers, comparable to using a keyboard and mouse. In terms of moving the user in virtual space, most respondents note the superiority of moving using the keyboard compared to controllers, since the latter require time to develop the necessary skills, and the process of moving causes various negative sensations (motion sickness, nausea, dizziness). This result suggests the relevance of a whole line of research related to the development of new software and hardware methods of moving in virtual reality.

In virtual reality, according to the majority of respondents, it is possible to study a virtual object more thoroughly from different angles, evaluate its properties, position and sizes (due to the effect of a three-dimensional image). However, some respondents noted that the insufficiently high resolution of virtual reality helmet displays reduces the detail of objects, they could look clearer on the monitor screen. The respondents note that with the use of virtual reality, they managed to place and interact with objects in a natural way, after training, some muscle memory was really developed.

Next, the proofs of the rendered hypotheses at the beginning of study will be considered.

4.1 H1—Negative influence of virtual reality

During the course of the exercises (moving objects and completing tasks in the virtual space of the mine), the participants of the experimental group actually encountered, to one degree or another, the negative influence of virtual reality. The group experienced negative feelings—nausea, motion sickness, dizziness, disorientation in space. The greater severity of these effects (Q1-Q4) when using VR is reflected in Table 2.

However, the negative experienced feelings did not prevent the group from successfully completing tasks in VR. The average values of the accuracy of solving problems, the number of errors when using VR are comparable to configurations based on the keyboard, mouse and monitor. The time of completing exercises in VR is longer, which, however, corresponds to performing actions in the real world, as when controlling the keyboard and mouse, some operations are carried out much faster due to the specifics of this control method and simplified interaction with objects by simply pressing keys.

Thus, the hypothesis is only half confirmed—virtual reality does have a negative impact on a person, but this effect does not prevent a person from successfully and effectively performing exercises in the process of professional training.

4.2 H2—Broadening of professional preparation opportunities

When using virtual reality in the first exercise, the participants of the experimental group, obtained higher results, in the accuracy of the task and made fewer mistakes (Table 2). In the second exercise, the superiority of VR configurations according to these metrics was minimal or absent due to the specifics of calculating the accuracy metric.

According to users’ subjective assessments, virtual technologies allow interacting with objects and studying them at a qualitatively new level, which leads to the development of not only knowledge about the correct sequence of actions, but also about the specifics of performing these actions, which leads to the development of the necessary muscle memory. This result is reflected in the Q8 metric of Table 2, which reflects the superiority of VR configurations. Moving in virtual reality during testing is still far from realistic, since it is carried out using controllers, but the use of adaptive running platforms will solve this problem as well (Caramenti et al. 2018).

The confirmation of the H2 hypothesis is based on the fact that the process of professional training is associated with the performance of point actions by the user with mechanics close to real. Only VR-based systems can provide such a way of interacting with objects. Table 2 confirms that when performing precisely point actions (moving objects), the VR configuration allows you to achieve better accuracy results and reduce the number of errors. As a result, the second hypothesis is fully confirmed, since the use of VR technologies expands the possibilities of professional training due to better and more realistic mechanics of interaction with objects, as well as (in the future) movement in space.

4.3 H3—Dynamics of changes in negative effects

An analysis of the assessments of the experimental group participants showed that after passing the tests, the assessment of the negative factors of virtual reality decreased: for nausea—by 49–100%, for motion sickness—by 58–86%, for dizziness—by 65–78%, for disorientation—by 61–70% (obtained based on a comparison of the first and last attempts by metrics Q1–Q4). The reduction in the negative impact of virtual reality is also confirmed by the improvement of quality metrics—with each repetition of the exercise, the accuracy of completing tasks increases, the passage time and the number of errors decreases, and also increases the value of metrics for assessing the convenience of interacting with virtual objects and the environment (Q5–Q8). Thus, the third hypothesis is fully confirmed based on the analysis of qualitative and quantitative metrics: in the process of interaction with virtual reality, due to adaptation, the severity of negative effects decreases in the learner.

4.4 Further studies

The present research allowed us to form a certain sample of data on the negative factors of the influence of virtual reality on the learning process. In the course of testing the hypotheses, certain conclusions were drawn about the effectiveness and problems of using virtual reality technologies in the process of professional training. However, this study is limited to the considered quantitative and qualitative metrics and a set of exercises and virtual scenes; therefore it does not cover all aspects of the question of the influence of virtual reality on a person.

The plan for further research includes the following areas:

• Expanding the list of quantitative metrics by adding sensors to the human body and taking new data (pulse and EEG), which will allow you to get a more complete picture of the impact of virtual reality on a person in various states (rest, activity, stress, and so on);

• Adding a configuration based on the use of an adaptive running platform, which will increase the realism of movement in virtual reality, realize simultaneous movement and interaction with objects, eliminate the development of incorrect patterns of behavior (when a person gets used to moving through controllers); this direction also includes the replacement of controllers with virtual reality gloves, which further increases the realism of interaction with virtual objects;

• Implementation of additional virtual scenes aimed at identifying new factors affecting a person in the process of working in virtual immersion: the magnitude of visual and auditory reactions to events, the quality and detail of virtual objects, the color characteristics of objects, the effect of illumination in the scene and the distance to objects on the accuracy of their perception of a person, and so on;

• Expansion of the list of metrics and the study of equipment parameters such as frame rate, screen resolution, image output delay, which will reveal additional factors of the negative impact of virtual reality.

Conducting research in these areas will allow identifying new patterns of the influence of virtual reality on a person, as well as to form a list of recommendations that will improve the effectiveness of professional training using such technologies.

5 Conclusion

The article considers the relevant problem of effective organization of professional training using virtual reality technologies on the example of the coal and mining industry. To solve it, it is necessary to conduct a comprehensive study of the process of human interaction with virtual reality, identify negative factors affecting it in the process of activity in the virtual space, as well as evaluate the effectiveness of the training process.

The study formulated three hypotheses covering the negative impact of virtual reality on a person, as well as the effectiveness of vocational training using these technologies. To confirm them, the necessary studies were carried out, including the following stages: analysis of existing quantitative and qualitative metrics of interaction with virtual reality, negative factors of VR influence on a person; formation of a methodology for evaluating and conducting experiments; development of the necessary software for virtual scenes; direct tests on the experimental group with the analysis and processing of the obtained results of quantitative and qualitative metrics.

The obtained results helped to partially or completely confirm the correctness of the established hypotheses about the presence of a negative impact of virtual reality on a person in the process of professional training, reducing this influence with the growth of user experience, as well as the possibility of improving the quality level of training due to the specifics of working in virtual space through VR technologies. To confirm the hypotheses put forward, statistical analysis of the collected experimental data was used, including data correlation studies and nonparametric Mann–Whitney and Kruskal–Wallis tests.

The obtained patterns of influence of virtual reality in the course of research can be applied in the process of organization of professional training in various subject areas, as well as in the application of VR technologies in distance learning. Reducing the severity of the identified negative factors from the use of these technologies will increase the immersiveness of the learning process and its effectiveness. Therefore, the direction of further research is to identify new negative VR factors and reduce their impact on humans, to test other approaches to interacting with virtual space to expand professional training opportunities (adaptive running platforms—for moving, VR gloves—for interacting with objects).