Keywords

1 Introduction

With the renewed interest in manned exploration of celestial bodies, first and foremost the Moon, robot-assisted surface exploration is going to play a major role in the near future in many space programs. In various cases, however, full robot autonomy is not a viable option within certain margins of safety, and higher level commands designed for high communication delays may be overly time consuming and complex. Some form of teleoperation has been shown to be possible in scenarios where the operator has to control the robotic platform from orbit with communication delays in the order of magnitude of a second or more. Therefore, teleoperation is deemed preferable whenever possible in space related tasks involving remote functionalities [1, 2]. Different space missions, such as ESA Haptics-1 [3], Kontur-2 [4] and Analog-1 [5], have investigated the feasibility of telemanipulation and telenavigation in microgravity conditions. In [6], a fictitious force feedback principle for high communication delays was developed for the Kontur-2 mission. A space qualified DLR force feedback joystick with two degrees of freedom (DoF) along with a car-like curvature and longitudinal velocity interface was used to access the three planar DoFs of the omni-directional rover. A comparable 2-DoF interface without force feedback was used in the Analog-1 mission in 2019 [7], which involved telenavigation as well as sample picking and placing with an earth-based rover from the ISS at >800 ms round-trip delay. The rover was navigated through relatively obstacle-free environment, such that force-feedback is of minor importance during telenavigation. To name an example in the foreseeable future, a similar interface to the one proposed in [7] is planned to be used in the upcoming Arches experiment [8] on mount Etna, albeit with both the pilot and the rover being based on the ground, and with no force feedback. However, some form of corrective feedback is necessary for telenavigation tasks in non-deterministic environments in the presence of communication delays, which otherwise may lead to performance deterioration. Previous studies [9], show the benefits of haptic force feedback in rover navigation in such conditions in terms of collision avoidance. However, introducing a closed loop force feedback can lead to instability at high delays. Different control principles as Routh-Hurwitz [10], Llewellyn approach [11] and Time Domain Passivity Control (TDPA, [12]) have been proposed to guarantee stability in delayed telenavigation with haptic feedback. In [13] and [6], the TDPA was extended for telemanipulation and different types of force feedbacks.

In consideration of these previous studies, we propose that a force feedback setup with a fitting TDPA control can bring more sensible benefits in a complex to navigate physical environment. Therefore, in this paper, we present a user study involving the DLR Joystick [14] and the DLR Lightweight Rover Unit (LRU, [15]) with the goal to evaluate the advantages of telenavigation with fictitious force feedback against telenavigation without force feedback. The TDPA is applied in this work due to its robustness to varying delay, packet loss and jitter. The main goal of this study is to investigate the effects of our force feedback setup on rover navigation in close proximity to physical obstacles.

2 Materials and Methods

2.1 Sample

The user study was conducted with 16 subjects (3 females, 13 males) with an average age of M = 25.2 yrs. (SD = 2.9 yrs.; range of 21 to 32 yrs.). All of them signed an informed consent form prior to the experimental session.

2.2 Apparatus

Rover. Subjects controlled the LRU wheeled mobile robot (WMR) [15], a prototypical rover system specifically designed for rough terrain. The LRU has 12 DoF, with four wheel actuators, four steering actuators, two series-elastic joints and two joints in the pan-tilt unit (which is equipped with a black-and-white stereo and a colour camera). The total weight is 30 kg with a maximal payload of 5 kg. Two battery packs allow for more than 120 min operation time. The maximal speed is 1.11 m/s. The stereo camera images are processed by performing a dense stereo matching using an FPGA implementation of the so-called Semi-Global Matching (SGM) algorithm [16]. Additionally, the colour camera images are mapped onto the resulting depth image for object detection. The resulting estimates are used for generating a danger map indicating insurmountable obstacles.

Joystick. The DLR’s Kontur-2 force feedback joystick [14] was used to drive the rover. The joystick has a 2-DoF, ±20\(^\circ \) workspace, a maximum force of 15 N and an update rate of 1 kHz. A car-like mapping of the 2 DoFs was implemented, i.e. longitudinal velocity was commanded by moving the joystick forwards and backwards, whereas curvature (steering the LRU’s front and back wheels) was commanded by lateral movements of the joystick. In order to navigate, the user has to press the dead-man button on the joystick. In order to switch the current movement direction of the LRU between forward and backward, the user has to press the lower side of the switch on top of the handle.

2.3 Force Feedback Controller

Figure 1 shows the signal flow diagram of a delayed bilateral teleoperator (BT), where \(T_1\) and \(T_2\) are the forward and backward delays, respectively. The coupling controller Ctrl ensures that the LRU (slave robot) follows the delayed master reference \(v_m^{del}\) and in turn generates a fictitious force \(F_f\), which is felt by the operator through the master haptic-device (DLR’s force feedback joystick) after a communication delay.

Fig. 1.
figure 1

Signal flow diagram of a bilateral teleoperator with fictitious force feedback

Full size image

Persuasive force-feedback from the virtual environment VE is produced via the direction dependent curvature polygons \(P_L\) and \(P_R\) (Fig. 2) overlapping with the danger map (Fig. 3) which is generated using stereo vision. The danger map associates to any pixel (x, y) in the vicinity of the rover a binary danger index D(x, y) (0 if there is no obstacle, 1 if an obstacle is present). The force components are computed according to:

$$\begin{aligned} F_{d} \propto \varSigma _{(x, y) \in P_{d}} D(x, y)\quad \text {for} \quad d = R, L \end{aligned}$$
(1)
$$\begin{aligned} F_B \propto \varSigma _{(x, y) \in P_R \cup P_L} D(x, y) = F_L + F_R \end{aligned}$$
(2)

An equally valid way of describing the calculation of the fictitious force feedback component is considering the obstacles and curvature polygons as sets containing the corresponding pixels on the danger map as elements. This way, the force feedback components can be computed using set operators. Figure 2 uses this notation to show a particular example with the calculation of force components for a specific obstacle and curvature polygon configuration. Notice that the curvature polygons are placed in a manner such that the subscripts of the variables F, P and O are consistent in the shown formulas, rather than respecting the actual right and left side in the rover coordinate frame. Such a system is prone to instability due to the energy generation by the delayed communication channel. In order to passivate the communication channel, Time-Domain-Passivity-Approach (TDPA) was implemented [6]. TDPA introduces adaptive virtual damping elements, both in a series and parallel fashion, at the master and slave side, to dissipate the exact amount of energy necessary to stabilize the overall system.

Fig. 2.
figure 2

An example of fictitious force component computation [6]

Full size image
Fig. 3.
figure 3

Screenshot of the user interface showcasing the danger map

Full size image

2.4 Experimental Task and Design

Participants had to drive the rover along or through different rock formations avoiding collisions (see Fig. 4). One complete experimental trial consisted of three subtasks: 1) navigating the rover through a narrow passage, 2) navigating along a curved rock formation as close as possible to the concave side, 3) navigating as close as possible along a convex boulder (see Fig. 5). Each trial was started from the same, predefined starting position and after having completed subtask 2, the rover was re-positioned by the experimenters to have an optimal starting position for the final subtask.

Fig. 4.
figure 4

LRU in experimental field

Full size image
Fig. 5.
figure 5

Schematic subtask representation

Full size image

The two experimental factors Force Feedback (FF vs. noFF) and Delay (0 ms vs. 800 ms) resulted in four experimental conditions. For each condition, there were two subsequent experimental trials, for a total of eight trials per subject. While the order of the two force feedback conditions was counterbalanced across subjects, these always started with the no delay condition and then proceeded with the 800 ms delay condition, as pre-tests showed that navigation with 800 ms delay was too demanding for inexperienced users.

2.5 Procedure

First, subjects were given instructions on the experimental task and procedure. They conducted the experiment at a table, sitting on a chair. The joystick was positioned to the right of the subject. Subjects were asked to adjust chair height so that their right arm rested comfortably on the padded arm rest of the joystick module. The experimental GUI (see Fig. 3) was displayed on a 23” monitor in front of the subjects. Since the participants controlled the rover remotely (i.e. from a separated room), the rover’s position and a danger map were displayed in the GUI together with a video stream shown in the upper left corner of the window. After having finished two training trials (with force feedback and without delay), the eight trials of the main experiment were started. In the room where the LRU was located, a technician and a supervisor managed the experiment parameters, and provided feedback about collisions and successful reaching of the intermediate milestones for each subtask.

2.6 Measures and Data Analysis

The metrics chosen to assess user performance are the number of actual collisions between rover and obstacles, task completion times, mean rover speed, mean and standard deviation of lateral forces on the master side of the control loop. Additionally, the centroid of all the odometry-measured path points followed by the LRU was used as a reference point in order to obtain the mean radius of curvature at which the subjects drove for subtask 2. For this subtask specifically, this metric can be used to estimate task fulfillment in terms of mean proximity of the followed path to the concave rock formation. For subtask 3, an analogous metric was computed based on the mean minimal distance for each path point from the line connecting the LRU’s average starting position for subtask 3 to the centroid of all recorded path points. Data was analysed using a repeated measures ANOVA (rmANOVA) with Force Feedback (FF vs. noFF), Delay (0 ms vs. 800 ms) and Subtask as within factors. See Table 1 for a result overview.

Table 1. Result overview; means, standard deviations (in parentheses) for all experimental conditions and significant ANOVA main effects.
Full size table

3 Results

While no significant effects were evident for the actual collisions, completion times tended to be longer when force feedback (FF) was activated compared to the noFF baseline, although the conventional level significance was not reached (p < .10). Similarly, there was a trend for the Subtask factor (p < .10). Specifically, completion times were longer for Subtask 1 (80.3 s) and Subtask 3 (87.4 s) compared to subtask 2 (67.2 s). A post-hoc contrast analysis, comparing the effect of FF on vs. off on completion times in each subtask revealed a (marginally) significant effect for subtask 3 only (t(15) = 2.09; p = .05), i.e. times were longer with FF compared to noFF. No significant differences were found for subtask 1 and 2. The mean speed of the rover was significantly reduced when FF was activated compared to the noFF condition (p < .001). The mean as well as the standard deviation (SD) of the lateral mean force was reduced when FF was provided at the joystick (both ps < .05). The task number has no observable effect on this metric, i.e. the positive effect of decreased mean and SD of force in the presence of FF was evident for all three subtasks. Finally, we checked the statistical power of the current study design since sample size was comparably small. Post-hoc statistical power analysis showed a power \(1-\beta =.99\) (well above .80 which is the desired probability) for the utilized design, sample and determined effect size.

4 Discussion

The fact that lateral force consistently decreases in the presence of FF for all subtasks indicates less overlap between the control polygon and obstacles on the danger map and therefore a safer navigation. This is to be expected, as the FF would exert forces on the input device inducing the subject to adjust the commanded trajectory in such a way that the overlap with obstacles is minimized. Despite the force feedback, the position drift of the TDPA brings a positive effect which was also found for measured force feedback in the telemanipulation setup of [17]. Since forces only appear in case of overlaps of the polygon with the obstacles, the position drift appears specifically in the presence thereof and prevents the overlap from increasing. In the absence of this feedback, the subjects are unhindered in commanding trajectories which intersect with obstacles.

The mean average velocity is consistently lower in the presence of FF. This indicates that the subjects tended to drive the rover more cautiously in this case. In fact, FF prevents the user to drive at higher velocities. The fact that the radius of curvature for subtask 2 tends to be higher in the presence of FF (ANOVA yields effect significance with p < .05), while the number of collision remains constant, shows that FF does provide the driver with information allowing for safe navigation even if closer to the outer rock formation. Interestingly, ANOVA shows no significant effect of the FF/noFF condition on completion time for subtask 1 and 2. When looking at this result, together with the reduced velocity in the presence of FF, this indicates that a more efficient path was followed using FF. Specifically, considering that the mean radius of curvature increases and mean velocity decreases in the presence of FF, the only way of explaining the absence of a significant difference in completion time is to infer that the subjects tended to change the commanded path curvature more often in the absence of FF, thus leading to a less efficient overall trajectory. In subtask 3, no significant effects of the FF condition were observed on the mean distance from the central rock formation. It would therefore seem that the FF/noFF condition has less bearing on navigation performance when confronted with convex structures, such as the one present in subtask 3. However, the lower incidence of lateral force in this subtask with FF, even though task completion was equivalent with or without FF, could be considered an index of safer navigation. These results show that some sensible benefits on rover navigation of using a force feedback setup with a fitting TDPA controller are measurable in a hard to navigate physical environment, even with sensible communication delays.