Keywords

1 Introduction

Robotic teleoperation, enabled by technological advancements, offers a unique opportunity to complete tasks remotely, avoiding potential dangers or inconveniences for the operator in cases where human operation is still required. As a result, areas of application include space operations, surgical procedures and a variety of other domains [1]. However, manually operating a robot from a remote environment poses unique challenges for the human operator e.g. in terms of workload, situational awareness or operator well-being [2]. As one potential solution, the robotic system can assist the operator during task execution through the integration of various levels of automation [3]. However, it is well known that even though high levels of automation allow for higher task efficiency, they may come at additional costs, leading to e.g. low situation awareness, over-reliance and an erosion of skills [4]. Therefore, there is an urgent need to address issues caused by high levels of automation. Here, haptic shared control approaches have been associated with the beneficial effects of automated systems, without causing strong automation-induced challenges [5]. Haptic shared control has been described as allowing “(...) both the human and the [automation] to exert forces on a control interface, of which its output (its position) remains the direct input to the controlled system.” ([6], p. 501). Hence, forces and movements applied by the operator and robotic system interact conjointly with each other during task completion [5]. Thus, haptic guidance forces can be used as virtual fixtures to guide the user along a predefined path or workspace [7]. Furthermore, redundant poses can be eliminated by limiting the operator’s degrees of freedom, resulting in increased task performance. Additionally, the stiffness of guidance forces can be adjusted to provide adequate guidance forces for various task demands.

The use of haptic guidance has been implemented in a variety of applications. It has been demonstrated, e.g., that haptic guidance improves completion time, error rate and distance from the ideal trajectory in a surgical spiral path following task [8]. Studies also indicated that haptic guidance should be generally implemented with higher stiffness values. For instance, [9] utilized a haptic virtual guidance fixture for a curve following task. They distinguished between no guidance and three different degrees of stiffness (“soft”, “medium”, “complete”), whereas “complete” guidance was associated with best results in terms of performance accuracy and error reduction. Moreover, the benefits of assistance functions are particularly evident in situations with high workload. Yet, only little research has been conducted to investigate possible effects of cognitive load on haptic guidance. Here, [10], e.g., reported that when haptic guidance is applied to a virtual vehicle steering task via torques applied to the steering wheel, a reduction in deviation from the centerline can be achieved. Additionally, they concluded that haptic guidance can be used effectively to mitigate the performance-degrading effects of additional cognitive load when the latter is induced via a secondary task.

So far, no research has been conducted to investigate how different types of haptic guidance with different stiffnesses and cognitive load interact and how it affects task performance as well as user experience. In the current work, these factors were investigated in a user study, conducted in a virtual environment with a 7-DoF haptic device.

2 Methods

2.1 Apparatus

A force feedback device with seven actuated DoF and active gravity compensation (lambda.7, Force Dimension) was used as I/O device (see Fig. 1, left). The device can be used within a comparably large workspace and produces high maximal forces with a high resolution. A HTC VIVE Pro Eye head- mounted display (HMD) was used for displaying the experimental simulation to the user (Resolution of \(1440 \times 1600\) pixels per eye; 90 Hz refresh rate). The virtual environment was run in the Unity video game engine. The haptic rendering and physics simulation was performed via a combination of multiple asynchronous real time processes. Among them, the “Voxel Pointshell” (VPS) algorithm with an update rate of 1 kHz was used [11]. The haptic guidance paradigm was realized using a predefined hard-coded virtual guidance fixture that actively attracted the participant onto the ideal path independent of trajectory progress. The ideal path was constructed from 19 waypoints distributed in relation to the object positions and a hard-coded safety margin. A Catmull-Rom spline algorithm was then used to interpolate and smooth the final path. The delta transformation between the current haptic device transformation and the projected haptic device transformation on the ideal path was used to estimate the current deviation from the optimal trajectory. The resulting quadratic distance was multiplied with the chosen stiffness to produce the forces and/or torques of the virtual fixture. A quadratic increase of forces and/or torques was implemented, since [12] e.g. reported that this paradigm leads to best human performance compared to other paradigms.

2.2 Sample, Experimental Setup and Tasks

Sample. N = 26 employees of the DLR voluntarily participated in the study (\(M_{\text {Age}}\) = 32.27 [SD = 10.72]; 3 females, 23 males). All subjects were right-handed and had normal or corrected to normal vision.

Experimental Setup. The haptic device was positioned at a predefined position on ground. Participants were seated in such a way that the haptic input device could be operated optimally with the right hand (Fig. 1, left). Seat height and distance to the device were adjusted individually, so that the device’s x-axis (with handle in null position) and the longitudinal forearm axis when holding the handle were aligned. A computer keyboard for responding to the secondary task was placed and fixated on a table next to the seat and was operated with the left hand. Finally, participants put on the HMD, individually adjusted head-strap and lens distance.

Fig. 1.
figure 1

Experimental Setup (left), VR Scene (middle), and Task Sequence (right)

Full size image

Experimental Tasks. In the experimental simulation, various static objects as well as a user-controlled object were visually and haptically displayed. An obstacle avoidance and a peg-in-hole insertion task had to be performed in this scenario (Fig. 1, middle). A formation of four pegs (with square bases) in the simulation was used for the obstacle avoidance sub-task. Participants had to move a peg from a standardized starting position through the obstacles in a pre-defined chronology (Fig. 1, right) and were instructed to prioritize collisions avoidance while also completing the task as quickly as possible. The respective next obstacle that had to be circumnavigated was highlighted in red. The respective next passage between this obstacle and the neighboring obstacle was also highlighted by changing the latter object’s color to blue. The completion of the task was indicated by a color change of the platform beneath the obstacles from grey to blue. No additional visual or auditory feedback was provided during task completion. After having completed the obstacle avoidance sub-task, participants were instructed to insert the controlled peg into a hole in the platform. A specific peg orientation was required for insertion due to the keyhole-like shape of the hole. Again, participants were asked to 1) prioritize contact force minimization during insertion and 2) to complete the task as quickly as possible. After having inserted the peg completely, the I/O device haptically guided the participants back to the starting position and the next trial was started. In some conditions, a secondary task had to be performed simultaneously with the above described primary task. There were four rectangular boxes displayed at the top, bottom, left and right sides of the experimental GUI. Randomly, one of these boxes changed its color from white to green and participants were instructed to respond as quickly and accurately as possible by pressing the corresponding arrow keys of the keyboard.

2.3 Experimental Design and Procedure

A secondary task paradigm was utilized to investigate the effects of additional Cognitive Load, i.e. there were conditions with vs. without secondary task. Four Haptic Guidance conditions were compared: 1) In a control condition, no haptic guidance was provided (C). 2) Haptic support for peg translation (T), i.e. the system guided movements along the trajectory in the dimensions X, Y, and Z by applying a specific stiffness gradient. 3) Haptic support for peg rotation (R). Thus, the system haptically assisted the degrees of freedom in \(X_{rot}\), \(Y_{rot}\), and \(Z_{rot}\) by generating stiffness gradients. 4) A combination of both (\(T+R\)). In all conditions, collisions between peg and obstacles were displayed haptically. Three different spring Stiffnesses were implemented for T and for R. A weak (\(K_1\), T: 3600 \(\text {N}/\text {m}^2\); R: 1.2 \(\text {N}*\text {m}/\text {rad}^2\)), a medium (\(K_2\), T: 33420 \(\text {N}/\text {m}^2\); R: 6.6 \(\text {N}*\text {m}/\text {rad}^2\)) and a high level (\(K_3\), T: 60000 \(\text {N}/\text {m}^2\); R: 12 \(\text {N}*\text {m}/\text {rad}^2\)) of stiffness were implemented. The specific values were selected based on a series of test trials conducted by the research team. Moreover, peg sizes and user’s viewpoints were varied, to explore whether findings generalize across different scenarios. For each condition, subjects started with a small peg (1.5 cm \(\times \) 1.5 cm \(\times \) 8 cm) and then performed the task with a large peg (2.55 cm \(\times \) 2.55 cm \(\times \) 12 cm). Also, there were two different viewpoints on the virtual scene, which were exactly mirroring each other. The perspective was randomly assigned to the first trial (small peg) and then mirrored for subsequent trial (large peg).

Experimental Design. A 2 (Cognitive Load, C) \(\times \) 4 (Haptic Guidance, H) \(\times \) 3 (Stiffness, \(K_i\)) within-subject experimental design was utilized. Within each H condition, the three different degrees of Stiffness (\(K_1\), \(K_2\) & \(K_3\)) were implemented. The orders of C, H and \(K_i\) conditions were counterbalanced across subjects. Each participant performed two trials for each type of Stiffness (1. Small, 2. Large Peg). Thus, 24 experimental trials had to be completed for both Cognitive Load conditions (=48 trials).

Procedure. Firstly, subjects completed a demographic questionnaire, read and signed an informed consent form. Secondly, participants were briefed on the study’s background, experimental tasks and hardware. Thirdly, the seating position and HMD were adjusted and the experiment was started. Prior to each Haptic Guidance condition, subjects completed a test trial. Subsequent to each Stiffness condition, subjects verbally indicated the usefulness of the Haptic Guidance. When having completed a Haptic Guidance condition, the HMD was taken off and an additional questionnaire was completed. Between the two Cognitive Load conditions, subjects had a 5-minute break.

2.4 Measures and Statistical Analysis

Time to Complete (TTC) and Collision Forces were recorded as objective performance measures. After each Stiffness trial, participants rated the Usefulness of the respective Haptic Guidance (“The haptic guidance was useful: Rate on a 5-point scale from disagree (1) to agree (5)”). Subsequent to each Haptic Guidance condition, subjects filled the NASA-TLX Workload questionnaire [13]. Furthermore, subjects’ Perceived Control was rated by two self-constructed items (“I felt in control of the system” and “I felt controlled by the system”; 5-point Likert-type scales ranging from disagree (1) to agree (5)).

Repeated measure ANOVAs (rmANOVA) with Cognitive Load, Haptic Guidance and Stiffness as within factors were performed on all measures. In case of non-sphericity, Greenhouse-Geisser (GG.) adjustments were made. Post-hoc comparisons were conducted with Bonferroni correction.

3 Results

3.1 Objective Performance Measures

Time to Complete (TTC). For the Obstacle Avoidance Task, a marginally significant main effect of Cognitive Load was evident (F(1, 25) = 4.03, p < .10), i.e. TTC tended to be longer for conditions with secondary task (M = 3.58 s; SD = 1.26 s) than for tasks without secondary task (M = 3.27 s; SD = 1.42 s). Furthermore, a significant Haptic Guidance \(\times \) Stiffness interaction effect was found (F(6, 150) = 7.46, p < .001). For all Stiffness conditions, \(T + R\) was associated with shorter TTCs than in the control group and R guidance. However, for \(K_2\) and \(K_3\), T guidance also yielded shorter TTCs than in the control group (see Fig. 2, left).

Fig. 2.
figure 2

Interaction effects between Haptic Condition and Stiffness for the time to complete during the obstacle avoidance task (left) and for the interaction effect between Haptic Condition and Cognitive Load for subjective control ratings (right). “T” = translational guidance, “R” = rotational guidance, “\(T + R\)” = translational + rotational guidance, “C” = control group. Error bars indicate 95% confidence interval.

Full size image

For the Peg-In-Hole Task, a significant Haptic Condition \(\times \) Stiffness interaction effect (F(2.66, 66.56) = 5.00, GG., p < .01) was found. For \(K_1\) no significant Haptic Condition effects could be identified; for \(K_2\), \(T + R\) led to shorter TTCs compared to the control group and T guidance. For \(K_3\), R guidance additionally reduced TTCs when compared to the control group.

Collision Force. For the Obstacle Avoidance Task rmANOVA indicated a significant interaction between Haptic Condition and Stiffness Condition (F(6, 150) = 8.67, p < .001). While no Haptic Guidance effects were evident with \(K_1\), \(T + R\) and T guidance resulted in lower collision forces than R guidance and the control group for \(K_2\) and \(K_3\). Regarding the Peg-in-Hole Task, no significant effects could be observed.

3.2 Subjective Ratings

Workload. A significant main effect of Cognitive Load on the raw NASA-TLX sum scores was found (F(1, 23) = 17.88, p < .001). As expected, Cognitive Load resulted in significantly higher ratings (M = 9.29; SD = 2.25; scale range: 1(very low) - 20 (very high)) compared to conditions without Cognitive Load (M = 7.95; SD = 2.22). Also, Haptic Guidance yielded a significant main effect (F(3, 69) = 10.70, p < .001): \(T + R\) guidance (M = 8.00; SD = 2.40) was associated with significantly lower ratings than R (M = 8.88; SD = 2.29) and the control condition (M = 9.24; SD = 2.03). Also, ratings in T (M = 8.34; SD = 2.13) were lower than in the control condition.

Perceived Control. RmANOVA indicated a significant interaction between Cognitive Load and Haptic Guidance (F(3, 69) = 6.32, p = .001), which revealed that in conditions with secondary task, the ratings for \(T + R\) as well as T were significantly higher compared to the conditions without secondary task (both ps = .001), see Fig. 2 (right).

Usefulness. RmANOVA revealed that there was a significant main effect of Haptic Guidance (F(2, 46) = 25.76, p < .001) and Stiffness (F(1.16, 26.76) = 23.91, GG., p < .001) on usefulness ratings. Overall, \(T + R\) guidance (M = 3.92; SD = .50) was rated as being most useful, followed by T guidance (M = 3.49; SD = .77) and lastly by R guidance (M = 2.99; SD = .50). For Stiffness, \(K_3\) (M = 3.71; SD = .47) and \(K_2\) (M = 3.65; SD = .50) were associated with the significantly highest usefulness scores compared to \(K_1\) (M = 3.02; SD = .85).

4 Discussion

In the present study, different implementations of haptic shared control were evaluated in a virtual environment with basic obstacle avoidance and insertion tasks. Specifically, human task performance and subjective experience when working with haptic support providing guidance for translational, rotational, or both motions and various stiffnesses were investigated in a user study. Altogether, clear evidence for improved task performance was found when haptic guidance for translations was provided. Given that stiffness was sufficiently high (\({\ge }33420\,\text {N}/\text {m}^2\)) obstacle avoidance tasks were completed faster and with significantly lower collision forces. During the peg-in-hole task, there were two subtasks: moving the peg into the correct pose for insertion and then insertion in contact itself. Here, higher stiffnesses (Translations: \({\ge }33420\,\text {N}/\text {m}^2\); Rotations: \({\ge }6.6\,\text {N}*\text {m}/\text {rad}^2\)), allowed faster task completion and the haptic guidance for all DoF was particularly beneficial. Rotational guidance only yielded improved completion times if a high stiffness was implemented (12 \(\text {N}*\text {m}/\text {rad}^2\)). There was a trend that cognitive load (introduced by a secondary task) led to increased completion times, indicating that participants tried to save cognitive resources by slowing down. Interestingly, however, the overall result pattern for haptic guidance and stiffness was evident independently from cognitive resource availability. The positive effects of haptic guidance were also reflected in the subjective ratings of participants: haptic guidance for all DoF was rated best in terms of workload and usefulness, followed by translational guidance and then by rotational guidance. Moreover, participants’ ratings of usefulness was more positive for medium or high stiffnesses. Not surprisingly, participants felt that they had less control over the system when haptic guidance was activated. However, under conditions of additional cognitive load, the feelings of control increased when being supported by haptic guidance with translational support. Subjects seemingly had the impression of having the dual task situation more under control with these types of assistance.

Conclusions. In this work, it has been shown that haptic guidance improves task performance and reduces workload. Medium or high levels of stiffness are preferable in terms of task performance and user experience. Furthermore, the study provides evidence that translational guidance leads to a stronger sense of control under conditions of higher cognitive load.

Additional research should be conducted to determine how the proposed haptic guidance system could be made more adaptive. For instance, it may be advantageous to combine haptic guidance with the concept of a haptic wall or significantly stronger guidance when approaching obstacles. Here, varying degrees of stiffness could be implemented at different points throughout the parkour. In narrow passages, a high degree of stiffness may be implemented, while the user retains greater flexibility when navigating around the obstacles. This may have a beneficial effect on subjective feelings of control, as the operator retains more control and the haptic guidance only intervenes when necessary.