Keywords

1 Introduction

Today, the industry is tackling with changing cutomer needs that leads to the increased need of mass-customisation and personification, and to the reconfiguration of production lines. In order to implement product customization and reconfigurability, there is a need to incorporate human dexterity and intelligence with machine accuracy and repeatability by developing human-robot collaboration (HRC) concepts. In order to keep the operator safe and overall system productive, there is a need to tackle safety risks for the operator. However, the continuous change, risks mitigation and training is expensive and time consuming.

Among the emerging technologies, Virtual Reality (VR) became prevalent in the gaming industry and the social community (E.g., the advent of metaverse). Principally, Reality-Virtuality Continuum consists of two environments category, whereas one defined as pure real environment and another as virtual environment. VR provides solely of virtual objects for user where it utilizes computer graphic simulation for creating objects where it could be seen with monitor-based displays, partially or fully immersive head-mounted display [1]. As these categories is expanding and more hardware and implementation are introduced, the term Extended Reality(XR) is defined to consist of all these hardware in Reality-Virtuality continuum. While the ICT sector is advancing in the use of XR, the manufacturing sector is hesitant due to technical maturity and cost of use.

This paper proceeds to study the technical feasibility of VR technology as a tool in the safety training process and assess industry readiness level with a HRC use case.

2 Related Work

The safety of an operator in close proximity of robot system is a challenge in industry, regarding interfaces, interaction, trust, and technical validation of these systems, several comprehensive studies were explored in [2,3,4,5]. Virtual reality has been used in assembly applications in manufacturing and consists of different categories such as simulation design, as discussed in the literature [6, 7]. To examine different possible interactions, a study was conducted by [12] to compare mouse-based, phantom haptic, two configurations with the markless motion capture method, and video-based training. The result demonstrated no notable differences between these five groups’ experiments. Another finding was that haptic feedback in case of the collision did not improve learning transfer. In parallel, Augmented Reality has been largely investigated in the same categories. Multiple researchers studied applications where the goal was to provide augmented information on 2D surfaces with projector-based and camera systems. These systems monitor the operator’s movements and dynamically project working areas on a table [8,9,10]. Various studies were conducted to contribute to training applications for providing information, instruction, and visual presentation. In the following sub-sections, literature on VR in training, and safety training is presented.

2.1 Virtual Reality Training

Due to the demand of the manufacturing industry, integrating digital manufacturing and technology is growing rapidly. The users are needed to be trained for changing environments while providing relevant information for the tasks at hand. As virtual reality can provide a safe environment for training purposes various studies have been carried out. Gorecky [11] targeted the two main challenges: training system design,,and training content generation from data management that is required for virtual training. They proposed a methodology consisting of VISTRA’s architect, knowledge platform, training simulator, and knowledge sharing center for the content generation of the virtual training system based on semantic technologies.

Training users in online mode (factory floor) is time-consuming and difficult. Therefore, game engines such as UNITY are playing a big role in the advancement of the training system. Pérez [12] implemented a training system, where lab environments such as 3D scanners were mapped into the real environment and simulated the kinematic and dynamic of the robot and VR could provide a promising solution for companies regarding costs, standardization, usability, and training time. In [13] study is conducted where the operator worked in close proximity of an industrial robot where VR was used to increase the immersiveness. From the first study, the survey result depicted that assembly tasks with their hands (VR controller) without the presence mass of components improved the feeling of presence and immersion. Later on Mastas et al. [14] conducted the experiment consisting of two techniques: passive, which aimed to prepare proactive human behavior by utilizing cognitive help via VR, and active approach where the intention was to examine robot adaptive behavior in case of deceleration of trajectories and modification of them.

2.2 Virtual Reality Safety Training

Virtual reality safety training has been studied in fewer industry sectors; however, they had common reasoning behind, reducing injuries concerning user safety while confronting hazardous events. The same concept of training, whereas operators are trained to act when facing hazardous event, and how the operator could react on such circumstances. Buttussi [15] conducted a study in the aviation safety domain for the door opening process. The result demonstrated the significant usability of VR headsets over printed materials. Additionally, the study displayed that VR headset applications led to more engagement and satisfaction compared to other methods. Moreover, a similar result was carried out in the study of effectiveness compared between traditional and VR training approaches in a generic environment of power plants regarding theoretical and practical learning purposes [16].

Among research communities in collaboration with companies, safety training was examined in the construction and mining industry, where operator’s safety could have been violated more frequently. Zheng [17] developed a use case that demonstrates the effectiveness of safety training with respect to memorizing critical points when facing hazardous situations in a construction site of urban cities. In addition, improving training contents of safety for construction professionals and workers are studied in [18] and performed empirically experiment that result in user’s identification of hazardous events. Additionally, recent research was carried out on developing a framework to study immersive learning in the safety training process of mine rescuers [19]. Their model proposed multiple criteria such as trainee experience with gaming and technology, in addition to the learning experience with real case scenarios, utilizing VR features, and understandability of VR usability. The study demonstrated the positive impact of such criteria in immersive VR-based training. Tichon in [20] performed comprehensive literature based on the effectiveness of virtual reality safety training in the mining industry regarding different skills such as problem-solving, decision making, cognition.

3 Research Methods, Material and Approach

The research methods in this study consist of the definition of a case study, and assessment of the approach with AHP method. The aim of the study is to include the acknowledgement of safety risks and guidelines for operator, and how to avoid or react in case of hazardous events. The VR training will guide and explain to the user what events can trigger the safety system and lead to stopping a robot. The overall goal is to reduce the production stops that origin from the user errors.

The test case is defined in this study relates to the HRC pilot line for assembly of a diesel engine with the industrial robot ABB IRB 4600. In this regard, a VR-based safety training is presented in our research laboratories to investigate the possibility of reducing user mistakes and hazardous events while facing robots in close proximity. The concept of this training was introduced in [21], and 2D snapshot of environment is depicted in Fig. 1.

In this study, a VR HMD with high resolution selected as user required to experience fully immersed environment with realistic detail of each component of laboratory. The 3D models were rendered carefully to represent replica of real components. For an instance, the engine’s CAD models were 3D scanned, and robot models were used from simulation software library with detail rendering. The users can interact with VR HMD controllers to interact with virtual components for movement and assembly, where virtual components contain game engine Physics features. In addition, it is required to walk in the full-scale simulation area to perceive safety distances and working areas.

Fig. 1.
figure 1

VR based safety training environment

Full size image

The hardware selected for this application are HTC VIVE Pro headset with resolution of 2880 x 1600 pixels, refresh rate of 90 Hz and field of view of 110 degrees, and gaming desktop with 32 GB of RAM, NVIDIA Geforce GTX 1060 graphic card. The reason for selecting wireless HMD is that the virtual environment representation is on 1:1 scale of laboratory to improve feel of presence of an operator. HTC VIVE pro controllers have haptic feedback feature, but this feature has not integrated to the system as psychological effect of that in training system requires investigations. The UNITY game engine v 2020.3 is utilized to create visual elements such as working area borders and User Interface such as instruction for acknowledging of different safety measures. For creating more realistic laboratory presentation, manufacturing simulation’s software Visual Components v4.4 and Blender adds-on for animation rendering of robot’s movements were used.

4 Assessment Approach

Investigating technology feasibility and integration of technical development for manufacturing follows the path between the research community and SMEs and finally adaptation of it in the industry sector. Regarding this, technical development should go through a process to get mature enough to be accepted in the industry. Tirmizi in [22] provided an assessment method to analyze use cases in the development process and steps they have taken toward maturity. Pöysäri [23] proposed a feasibility analysis concept of reconfigurable pilot lines. In this study, KPIs in the aforementioned studies were employed, which are important and relatable for the VR training concept and conduct a technology feasibility assessment. An assessment of this multi-criteria was carried out through the AHP method introduced by [24]. Performing this analysis could be beneficial in the decision-making of adaptation VR-based safety training based on these criterions.

At first, the definition of the overall goal is to evaluate safety training in a HRC environment. Two alternatives of training was chosen: traditional training (i.e., paper-based, video instruction, and instructor-based training) and VR based training using fully immersive VR HMD devices. Following the hierarchy Fig. 2, at the second level of criteria, twelve KPI’s are considered, which is respect to first level criteria, use case, and technical criteria. First, use case criteria focus on managerial level, and there are common factors that companies consider when using different solutions. These criteria are as follows: hardware cost, development cost, implementation cost, ergonomic, and lifecycle. Second, technical criteria focus on technical aspects consist of seven KPI’s where there are studied in [22, 23], and the definition originated in the context of reconfigurability in [25]. The definition of KPIs is included in Table 1.

A questionnaire with 10 participants from university researchers who are experts in industrial robotics and manufacturing has been conducted. Researchers were informed about the definition of VR application through the text and later by a demo video of application. This step taken toward clarification of application opportunities and demonstrated possible level of realistic representation of laboratory in VR HMD.

5 Results

The local weights of safety training represented in Table 2 including two criteria levels. The table provides the interpretation of the critical features that contributed to the final decision. The results indicate that technical criteria, with a rate of approximately 60 percent, are significantly influential. Whereas use case criteria account for the remaining proportion with approximately 40 percent. The second level criteria are devoted to the fundamental components constructing each use case and technical criteria. Regarding use case criteria, the development cost is the most crucial KPI at first rank, and Ergonomic takes the second rank, with 21.36% of the local wight. In the case of technical criteria, customization alongside modularity was at the highest rate. It is worth mentioning that customization, modularity, and availability account for more than 55% of the importance of local weight.

Table 1. Multi-criteria definition
Full size table
Fig. 2.
figure 2

AHP hierarchy

Full size image

Figure 3 provides an overview of the participants’ response distribution to the various KPIs. The hardware cost appears to be right-skewed, with three participants dedicating a weight of more than 0.2. This signifies the weight of hardware cost is relatively spread out with a median of about 0.1. However, 75% of the hardware cost survey results fall under 0.2, and the standard deviation is 0.1. As Table 2 depicted, the respondents allocated greater importance to development costs. This is also evident in the boxplot, the highest median of the use case criteria is associated with development cost, where 75% of responses are in the range of 0.18 to 0.42. Moreover, the implementation cost and lifecycle weights seem to experience less variability. Ultimately, the ergonomic criteria, the second local influential weight, has the median of 0.25 and upper quartile of 0.27.

Table 2. Estimated local weights of 1st and 2nd level criteria
Full size table

In technical criteria, the given weights of integrability, customization, support, and diagnosability are to some degree concentrated. The median of integrability, support, and diagnosability is comparably close to one another, 0.1, 0.8, 0.8, respectively. Customization criteria appear left-skewed, representing that the majority of participants have a consensus of great importance towards this measure. Such trend can be seen by the standard deviation of 0.05 and close proximately of the median of 0.22 and maximum value of 0.29. Additionally, the median of modularity and availability is about 0.18. This highlights that the experts have a moderately indistinguishable position concerning both criteria. However, on average, partakers tend to favor modularity compared to availability. The criteria of support follow a normal distribution. Finally, the specialists in the field of factory layout design yield outliers of diagnosability and integrability.

During the survey, participants were asked to provide an importance scale from 0 to 9 in pairwise comparison for both training methods regarding second level criterion. Afterward, the average local weight compared to each second level criteria, and the proportional ranks are depicted in Fig. 4. It can be seen that development cost and ergonomics are instrumental in the VR. Correspondingly, the proportion of hardware cost and lifecycle in the VR outweighs the traditional method. Nevertheless, partakers believe that implementation cost is more important in the traditional method, and the valuation of implementation costs for both training methods is insignificant. Furthermore, scalability and modularity are critical in the VR. Experts asserted that diagnosability and support are comparatively vital in the case of VR. Moreover, customization and integrability in the VR method are third and fourth. It is worth mentioning that eleven out of twelve local priorities were valued more in the VR-based training method.

Fig. 3.
figure 3

Average local criteria weights in upper and lower levels of hierarchy among participants

Full size image
Fig. 4.
figure 4

A comparison between traditional (red) and VR-based training methods (blue) based on average local proportion in respect to second level criteria (colour figure online)

Full size image

Ultimately, we conclude our assessment results by presenting global priority sensitivity. Generally, the VR training is appreciated over the traditional method with 65.22%. Seven out of ten experts tend to stress their interest in the VR-based method. One participant ranked both traditional and VR training methods nearly equivalent.

6 Conclusions

This study introduced a novel application of VR for safety training in human-robot collaboration, assessment method and indicative results of its technical feasibility, and user acceptance. This study is founded on the notion that in HRC, VR are a potentially innovative type of safety training. The results suggest that VR is a viable solution to design and train operators regarding required safety concerns in collaborative workcells. Technical criteria plays a vital role in investigating solutions with potential adaptation for industrial use. However, the survey results indicate the importance of availability, modularity, and customization of VR-based safety training solutions.

Similar to any study, this research has several limitations. The assessment is done by experts in human-robot collaboration involved in small-scale use case projects. Due to the Covid-19 pandemic, receiving survey results from the respondents was challenging, which limited our assessment to 10 experts. The larger studies are planned to be launched with different users such as managers, machinery safety experts, and shop-floor technicians can synergies different perspectives on the question.