Abstract
Kinesthetic interaction typically employs force-feedback devices for providing the kinesthetic input and feedback. However, the length of the mechanical arm limits the space that users can interact with. To overcome this challenge, a large control-display (CD) gain (>1) is often used to transfer a small movement of the arm to a large movement of the onscreen interaction point. Although a large gain is commonly used, its effects on task performance (e.g., task completion time and accuracy) and user experience in kinesthetic interaction remain unclear. In this study, we compared a large CD gain with the unit CD gain as the baseline in a task involving kinesthetic search. Our results showed that the large gain reduced task completion time at the cost of task accuracy. Two gains did not differ in their effects on perceived hand fatigue, naturalness, and pleasantness, but the large gain negatively influenced user confidence of successful task completion.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Kinesthetic interaction as a form of human-computer interaction (HCI) is based on applying force feedback to provide motion sensations in muscles, tendons, and joints [1]. There is an increasing number of kinesthetic applications in different fields, such as education [2], medical training and simulation [3].
Providing realistic force feedback requires dedicated devices such as haptic gloves [4], kinesthetic pens [5] or grounded force-feedback devices (e.g., Geomagic Touch [6]). Among them, force-feedback devices provide a reliable desktop interface with high-resolution forces (up to 1 kHz) [7]. A major limitation of force-feedback devices is that the length of the mechanical arm limits the interaction space [7]. A common solution is to scale a small motion of the mechanical arm to a larger motion of the onscreen haptic interaction point (HIP), i.e., employing a large control-display (CD) gain [8].
The concept of CD gain has been previously studied in the context of pointing devices such as the mouse, touchpad and handheld VR controllers. The results suggest that applying a high CD gain can help reduce task completion time [9, 10]. In the context of kinesthetic interactions, some studies suggested that the visual feedback provided by different CD gains can influence kinesthetic perception and sometimes even override the perception available through force feedback [11,12,13]. Further, while using a force-feedback device, applying a large CD gain leads to a mismatch between hand motions and HIP motions, which thus could potentially influence the user’s control of the HIP.
Previous studies have used different techniques to enable kinesthetic interactions in large virtual environments without directly using a large CD gain. Dominjon et al. [14] used the bubble technique which adjusts the HIP speed based on the relative positions of the HIP and its bubble to reach objects. Li et al. [15, 16] employed gaze modality to move the HIP for reaching remote targets. Both methods maintained the unit CD gain while touching objects.
Overall, there is an agreement that applying a large gain may influence kinesthetic interactions [14,15,16]. However, it is still not clear how different CD gains affect task measures such as task completion time, accuracy of interaction and user experience in real-world kinesthetic tasks. In order to fill this gap, we conducted an experiment involving kinesthetic search on a soft tissue. Kinesthetic search is a typical kinesthetic task we perform in the physical world. It requires the users to touch the object and move their fingers along the surface to detect textural and material abnormalities on or under the surface. In a computer-based kinesthetic search task, the user needs to move the HIP while applying appropriate inward force to detect anomalies and the precise control of the HIP is crucial for efficient and accurate interactions.
We evaluated two commonly used CD gains in kinesthetic search: a large CD gain (=3.25) determined by the size of the required virtual space was compared to the baseline unit CD gain (=1). We varied the types of the search area as an independent variable since the effects of the CD gain may be influenced by the interaction area.
We collected objective data (the search time, the number of lumps that the participants missed and the search pattern gathered from the movement data of the HIP) and subjective data (the perceived hand fatigue, naturalness, pleasantness and user confidence in finding all the lumps) to evaluate the two CD gains. The study focused on the below research questions in the context of kinesthetic search:
-
Are there differences in the task efficiency and search accuracy using two gains?
-
Are there differences in user experience using two gains?
The paper first introduces the experiment, following by the results and discussion.
2 Experiment
2.1 Selection of CD Gains
The explored soft tissue was a cuboid model (52 × 32 × 32 cm along the x-, y- and z-axes). The model was placed at the center of the virtual space and fully filled the screen of the display. The physical workspace of the force-feedback device used in the experiment was 16 × 12 × 12 cm [6].
The study compared two CD gains (high and default). The high gain was 3.25, determined by the ratio between the tissue size and the device workspace (i.e., 52/16). Thus, a 1 cm arm movement lead to a 3.25 cm HIP movement and the workspace was increased to 52 × 39 × 39 cm which could cover the dimension of the virtual tissue.
The default gain was 1 and thus the workspace was 16 × 12 × 12 cm. To explore the virtual space beyond this workspace, we employed gaze as the section mechanism to relocate the device workspace [16]. The user had to pull the mechanical arm backward to a reset position and gaze at the target area for 500 ms. The workspace would then lock to that area until the user repeated this process. Such a method allowed robust switching of the workspace and ensured that there were no accidental switches during the task. This selected mechanism is not relevant from the perspective of the experiment. All analyses (e.g., search time) pertained only to the period when the user touched the virtual tissue, avoiding any potential influence of this mechanism.
2.2 Experiment Design
A within-subject experiment was designed in a controlled laboratory setting. The task for the participants was to identify the number of lumps underneath a soft tissue.
We manipulated the types of the search area as an independent variable with two levels: four small areas or one large area (Fig. 1(B)). For the large area, the tissue (52 × 32 \( {\text{cm}} \), along the x- and y-axes) was divided into four areas with the size 26 × 16 \( {\text{cm}} \) each. The trial of searching the large area included only one area (size = 26 × 16 = 416 \( {\text{cm}}^{2} \)), and four trials as a task group covered the area of the whole tissue. For the small areas, the tissue was divided into 16 small areas with the size 13 × 8 cm each. To make the search size of all trials consistent, one trial of searching the small areas consisted of four randomly selected areas out of the 16 possible options (size = 13 × 8 × 4 = 416 \( {\text{cm}}^{2} \)), and four trials as another task group covered the whole tissue.
The sizes of these areas were selected based on the required search time to avoid a very long experiment. Simultaneously, they were used to examine the effects of the two CD gains in practical applications. The size of each small area was selected so that it could be covered by both workspaces of the two gains. In contrast, the large area could be covered by the workspace using the high gain but was beyond the workspace using the default gain. The user needed to relocate the workspace four times to fully search the large area (see Fig. 1(A) as an example).
The lump number for each trial was randomly selected from 1 to 4. For each task group with four trials, the total number were 10 (1 + 2 + 3 + 4 = 10). The lumps were sphere models. Since we are interested in examining user control by collecting the movement data of the HIP, all lumps were set as the same radius (0.3 cm) for simplicity. The lumps were randomly distributed (along the x- and y-axes) within the search areas, but placed at the fixed depth (1.5 cm) and were invisible to the participants.
Each participant needed to complete four task groups (2 gains × 2 types of the areas = 4 task groups) with 16 trials (4 task groups × 4 trials per group = 16 trials) and 40 lumps (4 task groups × 10 lumps per group = 40 lumps).
The haptics were developed using H3DAPI with OpenHaptics rendering system [17]. The stiffness of tissue and lumps were implemented by the linear spring law with different stiffness coefficients (tissue: 0.06 and lumps: 0.1) and the friction was implemented by the kinetic friction with the same friction coefficient (both: 0.01). The visual deformation was implemented by the Gauss function, linearly increased following the HIP depth. HIP was visualized as a sphere with 0.3 \( {\text{cm}} \) radius.
The participants were asked to input the number of lumps they found using a keyboard after each trial. The system checked and recorded the missing number and the search time. In addition, the system also logged HIP movement data along x-, y- and z-axes during the task. A 7-point Likert scale questionnaire was used to record the subjective data. User confidence were collected after each trial and other subjective data (hand fatigue, naturalness and pleasantness) were collected after each task group.
The participants signed an informed consent form and were asked to complete the tasks as accurately and quickly as possible. They were free to adopt their own strategy (e.g., horizontal and vertical searching) with a maximum of two full searches for each trial. In addition, no extra hand-rest equipment was provided. The order of the CD gains and the area types were counterbalanced among the participants.
2.3 Participants and Apparatus
24 participants were recruited from the local university community (16 women and 8 men), aged between 20 to 35 years (M = 26.17, SD = 4.26). Six participants had used a similar force-feedback device (1–2 times). An MSI GS63VR 7RF laptop was used as the host computer. We used a Samsung 245B monitor as the display, an EyeX [18] to track the gaze, a Touch X device [6] as the kinesthetic interface and a keyboard to input the participants’ answer, shown in Fig. 1(A). We employed H3DAPI [17] for haptics and Tobii SDK [18] for accessing the eye tracker.
3 Results
3.1 Objective Data
We first conducted the Shapiro–Wilk Normality test that all data were not normally distributed (all p < .001). Thus, we used the 2 × 2 (gains × area types) aligned rank transform (ART) repeated-measures non-parametric ANOVA [19] for the analysis. The Wilcoxon signed-rank test was used for the post hoc analysis. Table 1 shows the overall ART ANOVA results. We focus our analysis on the main effect of CD gains and its significant interaction effect with the area types.
Search Time:
We calculated the mean search time of four trials in each task group. The results showed that the high gain (M = 144.27, SD = 49.48) led to a shorter task completion time than the default gain (M = 209.93, SD = 63.86; Z = −4.200, p < .001). Figure 2(A) illustrates the interaction effect. In searching the large area, using the high gain (M = 112.48, SD = 44.59) led to approximately 47.7% shorter time than using the default gain (M = 215.26, SD = 74.22; Z = −4.286, p < .001). In searching small areas, using the high gain (M = 176.06, SD = 65.26) caused approximately 14.0% shorter time than using the default gain (M = 204.60, SD = 67.57; Z = −2.257, p = .025).
Missed Lumps:
We calculated the sum of the missed lumps for each task group. Figure 2(B) shows that the participants using the high gain (M = 1.98, SD = 1.19) missed more lumps than using the default gain (M = 0.90, SD = 0.77; Z = −3.426, p = .001).
Covered Area:
we calculated the proportion of the searched area based on the movement and the radius of HIP. Using the high gain (M = 83.29, SD = 4.96) caused searching a smaller area than using the default gain (M = 88.39, SD = 4.68; Z = −4.229, p < .001). Figure 2(C) shows that using the high gain (M = 78.02, SD = 7.37) led to searching a smaller area than using the default gain (M = 88.17, SD = 4.91; Z = −4.286, p < .001) in searching the large area. There was no difference in searching small areas. A participant’s pattern for searching a large area is shown in Fig. 2(E) as an example.
Search Depth:
The HIP stability in the search depth may directly affect the search accuracy. To evaluate the stability, we calculated the average absolute deviation value of the HIP data (along z-axis) at the lump depth (1.5 cm) for both gains. Figure 2(D) shows that the high gain (M = 0.46, SD = 0.11) caused a lower stability of the HIP than the default gain (M = 0.39, SD = 0.07; Z = −3.857, p < .001) in the search depth.
3.2 Subjective Data
The data were analyzed with the Wilcoxon signed-rank test (Fig. 3). There were no statistically significant differences between two gains, in terms of perceived hand fatigue, naturalness, pleasantness, as well as user confidence in searching small areas. For the large area, using the high gain (M = 4.64, SD = 1.13) led to less confidence than using the default gain (M = 5.41, SD = 0.76; Z = −3.312, p = .001).
4 Discussion
We experimentally examined the effect of CD gains on kinesthetic search. The results show that CD gains and the area types have significant effects on task performance.
4.1 Differences in Task Completion Time and Search Accuracy
This study focused on the comparison of two different CD gains, where the movement of the device arm led to the different amount of HIP movement (1× and 3.25× respectively). Although the kinesthetic interaction involves complex hand behaviors and interaction feedback, our results show that a large gain increases the movement speed of HIP and thus reduces the task completion time, consistent with the common effect of the CD gain in the pointing tasks using the mouse [9].
However, the search time while using the high gain was influenced by the area types (Fig. 2(A)). It can be understood if we consider the search strategy used by our participants. Participants typically adopted a strategy that involved horizontal or vertical sweeping motions (Fig. 2(E)). Searching a large area easily enabled the participants to perform fewer sweeping motions. Searching multiple smaller areas made them perform numerous sweeping motions, potentially leading to longer task times.
While using the default gain, irrespective of the area types, participants performed more sweeping motions and thus caused more search time than using the high gain.
The results on the search accuracy presents a different picture. Regardless of the area types, using the high gain made participants miss more lumps than using the default gain (see Fig. 2(B)). There may be two explanations for this phenomenon. First, while using the high gain, the participants searched less area than using the default gain (Fig. 2(C)). Thus, the participants had a higher probability of missing the lumps using the high gain. Second, the lumps were fixed at the same depth inside the tissue. To find the lumps effectively, the participants had to maintain a constant depth of the HIP that could optimally touch the lumps while performing the sweeping motions. A more stable HIP depth presents better probability to find the lumps. Our result (Fig. 2(D)) demonstrated that using the high gain causes an increased variability in the HIP depth than using the default gain. Previous studies show that hand stability degrades under the stress of the force [20] and fatigue [21]. For the high gain, the stability issues may be amplified due to the scaling motion, and thus resulted in lower search accuracy.
4.2 Difference in User Experience
CD gain can potentially affect user experience, such as ease of use and pleasantness, in some HCI applications (e.g., [10]). Surprisingly, we did not find any difference between the two gain conditions in kinesthetic search, in terms of naturalness, pleasantness and hand fatigue. User confidence was influenced by two gains. Participants were generally less confident in finding all lumps while using the high gain, specifically while searching a large area. They likely had perceived the limited control over the HIP movement and were aware that they missed many areas. Using the default gain made participants more accurate in finding all lumps and subjectively more confident.
4.3 Limitations and Future Studies
This study has a few limitations. First, we examined two commonly used CD gains. Technically, the CD gain values that lie between them are rarely used due to the unsuitable workspace. Two levels (high and low) could sufficiently examine the general effect of the CD gain. However, a very large gain (i.e., the resulted workspace is much larger than the required space size) may cause different user performances (e.g., increase the task completion time, like [10]). Future work may examine this aspect.
Second, we used constant gains along x-, y- and z-axes. Dynamic gains were proposed for the pointing tasks (e.g., [8, 22]). However, their feasibilities for kinesthetic interaction are unknown. Dynamic gains (e.g., velocity-based) may lead to dynamic kinesthetic feedback and affect touch perception. Further, different CD gains could be potentially applied along the different axes. These should be studied further.
Third, the experiment involved a simple cuboid model with a flat surface. Practical applications may include models with irregular shapes and uneven surfaces (e.g., a heart model). The flat surface was a simple model that we could use to examine the effects of CD gains. Future work should test how results differ for complex models.
Fourth, we focused on kinesthetic search, a specific type of kinesthetic interaction. The CD gains may have different effects on different kinesthetic tasks, such as weight perception [11, 13]. Future work could examine CD gains in other kinesthetic tasks.
Fifth, this study included a short-term evaluation with new users. A prolonged usage or recruiting users such as medical professionals who are familiar with kinesthetic search may lead to different results. We propose these for the future research.
5 Conclusion
This study investigated the effects of CD gains on kinesthetic search. The experiment shows that a large gain improves task efficiency at the cost of user control and thus search accuracy. Our result experimentally demonstrates the significance to maintain the unit CD gain for accurate kinesthetic interaction. In addition, the findings of the study increase theoretical understanding of the CD gain effects on the task performance and user experience, which provide an experimental basis for designing new interaction techniques based on the CD gain for efficient and accurate kinesthetic interaction.
References
El Saddik, A., Orozco, M., Eid, M., Cha, J.: Haptics Technologies. Springer, Berlin (2011). https://doi.org/10.1007/978-3-642-22658-8
Grønbæk, K., Iversen, O.S., Kortbek, K.J., Nielsen, K.R., Aagaard, L.: Interactive floor support for kinesthetic interaction in children learning environments. In: Baranauskas, C., Palanque, P., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4663, pp. 361–375. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74800-7_32
Bielser, D., Gross, MH.: Interactive simulation of surgical cuts. In: PCCGA, pp. 116–442 (2000)
HaptX. https://www.haptx.com. Accessed 1 Dec 2019
Kamuro, S., Minamizawa, K., Tachi, S.: An ungrounded pen-shaped kinesthetic display: device construction and applications. In: WHC, pp. 557–567 (2011)
3D Systems. https://www.3dsystems.com. Accessed 1 Dec 2019
Massie, T.H., Salisbury, J.K.: The PHANToM haptic interface. In: DSC, vol. 55, no. 1 (1994)
Argelaguet, F., Andújar, C.: A survey of 3D object selection techniques for virtual environments. Comput. Graph. 37(3), 121–136 (2013)
Casiez, G., Vogel, D., Balakrishnan, R., Cockburn, A.: The impact of control-display gain on user performance in pointing tasks. J. CHI 23(3), 215–250 (2008)
Kwon, S., Choi, E., Chung, M.K.: Effect of control-to-display gain and movement direction of information spaces on the usability of navigation on small touch-screen interfaces using tap-n-drag. Ind. Ergon. 41(3), 322–330 (2011)
Dominjon, L., Richard, P., Fre, L.C., Richir, S.: Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments. In: Proceeding of IEEE VR (2005)
Li, M., et al.: Evaluation of pseudo-haptic interactions with soft objects in virtual environments. PLoS One 11(6), e0157681 (2016)
Samad, M., Gatti, E., Hermes, A., Benko, H., Parise, C.: Pseudo-haptic weight: changing the perceived weight of virtual objects by manipulating control-display ratio. In: CHI, Paper 320 (2019)
Dominjon, L., Lécuyer, A., Burkhardt, J.M., Barroso, G.A., Richir, S.: The “bubble” technique: interacting with large virtual environments using haptic devices with limited workspace. In: WHC, pp. 639–640 (2005)
Li, Z., Akkil, D., Raisamo, R.: Gaze augmented hand-based kinesthetic interaction: what you see is what you feel. IEEE Trans. Haptics 12(2), 114–127 (2019)
Li, Z., Akkil, D., Raisamo, R.: Gaze-based kinaesthetic interaction for virtual reality. Interact. Comput. 32, 17–32 (2020)
H3DAPI. http://www.h3dapi.org. Accessed 1 Dec 2019
TOBII. https://www.tobii.com/. Accessed 1 Dec 2019
Wobbrock, J.O., Findlater, L., Gergle, D., Higgins, J.J.: The aligned rank transform for nonparametric factorial analyses using only ANOVA procedures. In: CHI, pp. 143–146 (2011)
Borg, G., Sjöberg, H.: The variation of hand steadiness with physical stress. Motor Behav. 13(2), 110–116 (1981)
Gates, D.H., Dingwell, J.B.: The effects of muscle fatigue and movement height on movement stability and variability. Exp. Brain Res. 209(4), 525–536 (2011). https://doi.org/10.1007/s00221-011-2580-8
Wobbrock, J.O., Fogarty, J., Liu, S.Y., Kimuro, S., Harada, S.: The angle mouse: target-agnostic dynamic gain adjustment based on angular deviation. In: CHI, pp. 1401–1410 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this paper
Cite this paper
Li, Z., Akkil, D., Raisamo, R. (2020). The Impact of Control-Display Gain in Kinesthetic Search. In: Nisky, I., Hartcher-O’Brien, J., Wiertlewski, M., Smeets, J. (eds) Haptics: Science, Technology, Applications. EuroHaptics 2020. Lecture Notes in Computer Science(), vol 12272. Springer, Cham. https://doi.org/10.1007/978-3-030-58147-3_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-58147-3_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58146-6
Online ISBN: 978-3-030-58147-3
eBook Packages: Computer ScienceComputer Science (R0)