Keywords

1 Introduction

Kinesthetic interaction as a form of human-computer interaction (HCI) is based on applying force feedback to provide motion sensations in muscles, tendons, and joints [1]. There is an increasing number of kinesthetic applications in different fields, such as education [2], medical training and simulation [3].

Providing realistic force feedback requires dedicated devices such as haptic gloves [4], kinesthetic pens [5] or grounded force-feedback devices (e.g., Geomagic Touch [6]). Among them, force-feedback devices provide a reliable desktop interface with high-resolution forces (up to 1 kHz) [7]. A major limitation of force-feedback devices is that the length of the mechanical arm limits the interaction space [7]. A common solution is to scale a small motion of the mechanical arm to a larger motion of the onscreen haptic interaction point (HIP), i.e., employing a large control-display (CD) gain [8].

The concept of CD gain has been previously studied in the context of pointing devices such as the mouse, touchpad and handheld VR controllers. The results suggest that applying a high CD gain can help reduce task completion time [9, 10]. In the context of kinesthetic interactions, some studies suggested that the visual feedback provided by different CD gains can influence kinesthetic perception and sometimes even override the perception available through force feedback [11,12,13]. Further, while using a force-feedback device, applying a large CD gain leads to a mismatch between hand motions and HIP motions, which thus could potentially influence the user’s control of the HIP.

Previous studies have used different techniques to enable kinesthetic interactions in large virtual environments without directly using a large CD gain. Dominjon et al. [14] used the bubble technique which adjusts the HIP speed based on the relative positions of the HIP and its bubble to reach objects. Li et al. [15, 16] employed gaze modality to move the HIP for reaching remote targets. Both methods maintained the unit CD gain while touching objects.

Overall, there is an agreement that applying a large gain may influence kinesthetic interactions [14,15,16]. However, it is still not clear how different CD gains affect task measures such as task completion time, accuracy of interaction and user experience in real-world kinesthetic tasks. In order to fill this gap, we conducted an experiment involving kinesthetic search on a soft tissue. Kinesthetic search is a typical kinesthetic task we perform in the physical world. It requires the users to touch the object and move their fingers along the surface to detect textural and material abnormalities on or under the surface. In a computer-based kinesthetic search task, the user needs to move the HIP while applying appropriate inward force to detect anomalies and the precise control of the HIP is crucial for efficient and accurate interactions.

We evaluated two commonly used CD gains in kinesthetic search: a large CD gain (=3.25) determined by the size of the required virtual space was compared to the baseline unit CD gain (=1). We varied the types of the search area as an independent variable since the effects of the CD gain may be influenced by the interaction area.

We collected objective data (the search time, the number of lumps that the participants missed and the search pattern gathered from the movement data of the HIP) and subjective data (the perceived hand fatigue, naturalness, pleasantness and user confidence in finding all the lumps) to evaluate the two CD gains. The study focused on the below research questions in the context of kinesthetic search:

  • Are there differences in the task efficiency and search accuracy using two gains?

  • Are there differences in user experience using two gains?

The paper first introduces the experiment, following by the results and discussion.

2 Experiment

2.1 Selection of CD Gains

The explored soft tissue was a cuboid model (52 × 32 × 32 cm along the x-, y- and z-axes). The model was placed at the center of the virtual space and fully filled the screen of the display. The physical workspace of the force-feedback device used in the experiment was 16 × 12 × 12 cm [6].

The study compared two CD gains (high and default). The high gain was 3.25, determined by the ratio between the tissue size and the device workspace (i.e., 52/16). Thus, a 1 cm arm movement lead to a 3.25 cm HIP movement and the workspace was increased to 52 × 39 × 39 cm which could cover the dimension of the virtual tissue.

The default gain was 1 and thus the workspace was 16 × 12 × 12 cm. To explore the virtual space beyond this workspace, we employed gaze as the section mechanism to relocate the device workspace [16]. The user had to pull the mechanical arm backward to a reset position and gaze at the target area for 500 ms. The workspace would then lock to that area until the user repeated this process. Such a method allowed robust switching of the workspace and ensured that there were no accidental switches during the task. This selected mechanism is not relevant from the perspective of the experiment. All analyses (e.g., search time) pertained only to the period when the user touched the virtual tissue, avoiding any potential influence of this mechanism.

2.2 Experiment Design

A within-subject experiment was designed in a controlled laboratory setting. The task for the participants was to identify the number of lumps underneath a soft tissue.

We manipulated the types of the search area as an independent variable with two levels: four small areas or one large area (Fig. 1(B)). For the large area, the tissue (52 × 32 \( {\text{cm}} \), along the x- and y-axes) was divided into four areas with the size 26 × 16 \( {\text{cm}} \) each. The trial of searching the large area included only one area (size = 26 × 16 = 416 \( {\text{cm}}^{2} \)), and four trials as a task group covered the area of the whole tissue. For the small areas, the tissue was divided into 16 small areas with the size 13 × 8 cm each. To make the search size of all trials consistent, one trial of searching the small areas consisted of four randomly selected areas out of the 16 possible options (size = 13 × 8 × 4 = 416 \( {\text{cm}}^{2} \)), and four trials as another task group covered the whole tissue.

Fig. 1.
figure 1

(A) shows the experiment environment. The display shows an example of using the default CD gain to search a large area. The device workspace with the white boundary switches to the bottom right part of the large area where the user gazes at. (B) shows the area types.

Full size image

The sizes of these areas were selected based on the required search time to avoid a very long experiment. Simultaneously, they were used to examine the effects of the two CD gains in practical applications. The size of each small area was selected so that it could be covered by both workspaces of the two gains. In contrast, the large area could be covered by the workspace using the high gain but was beyond the workspace using the default gain. The user needed to relocate the workspace four times to fully search the large area (see Fig. 1(A) as an example).

The lump number for each trial was randomly selected from 1 to 4. For each task group with four trials, the total number were 10 (1 + 2 + 3 + 4 = 10). The lumps were sphere models. Since we are interested in examining user control by collecting the movement data of the HIP, all lumps were set as the same radius (0.3 cm) for simplicity. The lumps were randomly distributed (along the x- and y-axes) within the search areas, but placed at the fixed depth (1.5 cm) and were invisible to the participants.

Each participant needed to complete four task groups (2 gains × 2 types of the areas = 4 task groups) with 16 trials (4 task groups × 4 trials per group = 16 trials) and 40 lumps (4 task groups × 10 lumps per group = 40 lumps).

The haptics were developed using H3DAPI with OpenHaptics rendering system [17]. The stiffness of tissue and lumps were implemented by the linear spring law with different stiffness coefficients (tissue: 0.06 and lumps: 0.1) and the friction was implemented by the kinetic friction with the same friction coefficient (both: 0.01). The visual deformation was implemented by the Gauss function, linearly increased following the HIP depth. HIP was visualized as a sphere with 0.3 \( {\text{cm}} \) radius.

The participants were asked to input the number of lumps they found using a keyboard after each trial. The system checked and recorded the missing number and the search time. In addition, the system also logged HIP movement data along x-, y- and z-axes during the task. A 7-point Likert scale questionnaire was used to record the subjective data. User confidence were collected after each trial and other subjective data (hand fatigue, naturalness and pleasantness) were collected after each task group.

The participants signed an informed consent form and were asked to complete the tasks as accurately and quickly as possible. They were free to adopt their own strategy (e.g., horizontal and vertical searching) with a maximum of two full searches for each trial. In addition, no extra hand-rest equipment was provided. The order of the CD gains and the area types were counterbalanced among the participants.

2.3 Participants and Apparatus

24 participants were recruited from the local university community (16 women and 8 men), aged between 20 to 35 years (M = 26.17, SD = 4.26). Six participants had used a similar force-feedback device (1–2 times). An MSI GS63VR 7RF laptop was used as the host computer. We used a Samsung 245B monitor as the display, an EyeX [18] to track the gaze, a Touch X device [6] as the kinesthetic interface and a keyboard to input the participants’ answer, shown in Fig. 1(A). We employed H3DAPI [17] for haptics and Tobii SDK [18] for accessing the eye tracker.

3 Results

3.1 Objective Data

We first conducted the Shapiro–Wilk Normality test that all data were not normally distributed (all p < .001). Thus, we used the 2 × 2 (gains × area types) aligned rank transform (ART) repeated-measures non-parametric ANOVA [19] for the analysis. The Wilcoxon signed-rank test was used for the post hoc analysis. Table 1 shows the overall ART ANOVA results. We focus our analysis on the main effect of CD gains and its significant interaction effect with the area types.

Table 1. Tests of within-subject effects on the objective data (significant values are in bold).
Full size table

Search Time:

We calculated the mean search time of four trials in each task group. The results showed that the high gain (M = 144.27, SD = 49.48) led to a shorter task completion time than the default gain (M = 209.93, SD = 63.86; Z = −4.200, p < .001). Figure 2(A) illustrates the interaction effect. In searching the large area, using the high gain (M = 112.48, SD = 44.59) led to approximately 47.7% shorter time than using the default gain (M = 215.26, SD = 74.22; Z = −4.286, p < .001). In searching small areas, using the high gain (M = 176.06, SD = 65.26) caused approximately 14.0% shorter time than using the default gain (M = 204.60, SD = 67.57; Z = −2.257, p = .025).

Fig. 2.
figure 2

(A) shows the search times based on the gains and the area types (i.e., large and small); (B) shows the number of the missed lumps based on two gains; (C) shows the area proportion the participants searched based on the gains and the area types; (D) shows the average absolute deviation of the HIP depth based on two gains; (E) shows a participant’s pattern for searching a large area. The line in the boxplot is the median value and the cross mark is the mean value.

Full size image

Missed Lumps:

We calculated the sum of the missed lumps for each task group. Figure 2(B) shows that the participants using the high gain (M = 1.98, SD = 1.19) missed more lumps than using the default gain (M = 0.90, SD = 0.77; Z = −3.426, p = .001).

Covered Area:

we calculated the proportion of the searched area based on the movement and the radius of HIP. Using the high gain (M = 83.29, SD = 4.96) caused searching a smaller area than using the default gain (M = 88.39, SD = 4.68; Z = −4.229, p < .001). Figure 2(C) shows that using the high gain (M = 78.02, SD = 7.37) led to searching a smaller area than using the default gain (M = 88.17, SD = 4.91; Z = −4.286, p < .001) in searching the large area. There was no difference in searching small areas. A participant’s pattern for searching a large area is shown in Fig. 2(E) as an example.

Search Depth:

The HIP stability in the search depth may directly affect the search accuracy. To evaluate the stability, we calculated the average absolute deviation value of the HIP data (along z-axis) at the lump depth (1.5 cm) for both gains. Figure 2(D) shows that the high gain (M = 0.46, SD = 0.11) caused a lower stability of the HIP than the default gain (M = 0.39, SD = 0.07; Z = −3.857, p < .001) in the search depth.

3.2 Subjective Data

The data were analyzed with the Wilcoxon signed-rank test (Fig. 3). There were no statistically significant differences between two gains, in terms of perceived hand fatigue, naturalness, pleasantness, as well as user confidence in searching small areas. For the large area, using the high gain (M = 4.64, SD = 1.13) led to less confidence than using the default gain (M = 5.41, SD = 0.76; Z  = −3.312, p = .001).

Fig. 3.
figure 3

Subjective results of the study (a higher value is better).

Full size image

4 Discussion

We experimentally examined the effect of CD gains on kinesthetic search. The results show that CD gains and the area types have significant effects on task performance.

4.1 Differences in Task Completion Time and Search Accuracy

This study focused on the comparison of two different CD gains, where the movement of the device arm led to the different amount of HIP movement (1× and 3.25× respectively). Although the kinesthetic interaction involves complex hand behaviors and interaction feedback, our results show that a large gain increases the movement speed of HIP and thus reduces the task completion time, consistent with the common effect of the CD gain in the pointing tasks using the mouse [9].

However, the search time while using the high gain was influenced by the area types (Fig. 2(A)). It can be understood if we consider the search strategy used by our participants. Participants typically adopted a strategy that involved horizontal or vertical sweeping motions (Fig. 2(E)). Searching a large area easily enabled the participants to perform fewer sweeping motions. Searching multiple smaller areas made them perform numerous sweeping motions, potentially leading to longer task times.

While using the default gain, irrespective of the area types, participants performed more sweeping motions and thus caused more search time than using the high gain.

The results on the search accuracy presents a different picture. Regardless of the area types, using the high gain made participants miss more lumps than using the default gain (see Fig. 2(B)). There may be two explanations for this phenomenon. First, while using the high gain, the participants searched less area than using the default gain (Fig. 2(C)). Thus, the participants had a higher probability of missing the lumps using the high gain. Second, the lumps were fixed at the same depth inside the tissue. To find the lumps effectively, the participants had to maintain a constant depth of the HIP that could optimally touch the lumps while performing the sweeping motions. A more stable HIP depth presents better probability to find the lumps. Our result (Fig. 2(D)) demonstrated that using the high gain causes an increased variability in the HIP depth than using the default gain. Previous studies show that hand stability degrades under the stress of the force [20] and fatigue [21]. For the high gain, the stability issues may be amplified due to the scaling motion, and thus resulted in lower search accuracy.

4.2 Difference in User Experience

CD gain can potentially affect user experience, such as ease of use and pleasantness, in some HCI applications (e.g., [10]). Surprisingly, we did not find any difference between the two gain conditions in kinesthetic search, in terms of naturalness, pleasantness and hand fatigue. User confidence was influenced by two gains. Participants were generally less confident in finding all lumps while using the high gain, specifically while searching a large area. They likely had perceived the limited control over the HIP movement and were aware that they missed many areas. Using the default gain made participants more accurate in finding all lumps and subjectively more confident.

4.3 Limitations and Future Studies

This study has a few limitations. First, we examined two commonly used CD gains. Technically, the CD gain values that lie between them are rarely used due to the unsuitable workspace. Two levels (high and low) could sufficiently examine the general effect of the CD gain. However, a very large gain (i.e., the resulted workspace is much larger than the required space size) may cause different user performances (e.g., increase the task completion time, like [10]). Future work may examine this aspect.

Second, we used constant gains along x-, y- and z-axes. Dynamic gains were proposed for the pointing tasks (e.g., [8, 22]). However, their feasibilities for kinesthetic interaction are unknown. Dynamic gains (e.g., velocity-based) may lead to dynamic kinesthetic feedback and affect touch perception. Further, different CD gains could be potentially applied along the different axes. These should be studied further.

Third, the experiment involved a simple cuboid model with a flat surface. Practical applications may include models with irregular shapes and uneven surfaces (e.g., a heart model). The flat surface was a simple model that we could use to examine the effects of CD gains. Future work should test how results differ for complex models.

Fourth, we focused on kinesthetic search, a specific type of kinesthetic interaction. The CD gains may have different effects on different kinesthetic tasks, such as weight perception [11, 13]. Future work could examine CD gains in other kinesthetic tasks.

Fifth, this study included a short-term evaluation with new users. A prolonged usage or recruiting users such as medical professionals who are familiar with kinesthetic search may lead to different results. We propose these for the future research.

5 Conclusion

This study investigated the effects of CD gains on kinesthetic search. The experiment shows that a large gain improves task efficiency at the cost of user control and thus search accuracy. Our result experimentally demonstrates the significance to maintain the unit CD gain for accurate kinesthetic interaction. In addition, the findings of the study increase theoretical understanding of the CD gain effects on the task performance and user experience, which provide an experimental basis for designing new interaction techniques based on the CD gain for efficient and accurate kinesthetic interaction.