Surface Roughness Judgment During Finger Exploration Is Changeable by Visual Oscillations

Abstract

Previously, we proposed a pseudo-haptic method of changing the perceived roughness of virtual textured surfaces represented by vibrational feedback during pen-surface interaction. This study extends the method to finger-surface interactions in order to change the roughness judgment of real surface. Users watched computer-generated visual oscillations of the contact point while exploring the texture using bare fingers. The user study showed that our method could modify roughness judgment of real textured surfaces.

1 Introduction

Tactile texture rendering of virtual surfaces is important for providing the user with a highly realistic interaction experience with virtual objects. Such realistic tactile texture rendering of virtual object surfaces usually require some kind of electromechanical devices.

Fig. 1.

We propose a method for modulating the roughness perception of real textured surfaces touched by the users using their bare fingers.

Recently, pseudo-haptics has attracted attention as a light-weight haptic presentation approach. The pseudo-haptic sensation [3, 7] is produced by an appropriate sensory inconsistency between the physical movement of the body and the observed movement of a virtual pointer [4]. As for texture perception, some studies attempted to generate texture perception using the pseudo-haptic effects without sophisticated haptic devices [1, 2, 5, 9, 10]. Previously, we proposed a pseudo-haptic method for modulating the vibrotactile roughness of virtual surfaces during pen-surface interactions, and our user study showed the effectiveness of the proposed method [9]. In the method, users watched a visually perturbed contact point. However, since the proposed method assumed pen-surface interactions, users could not sense temperature or friction of the virtual surface.

In the present study, we attempt to apply the same method to finger-surface interactions. In that environment, the users watch the visually perturbed contact point while exploring real surfaces with their bare fingers (see Fig. 1). The research question of this research is to investigate whether the method can change the roughness judgment of the real surface during finger-surface interaction.

2 Concept

2.1 Pseudo-haptic Method Using Visual Oscillations

This study adopted the same pseudo-haptic method using visual oscillation as [9]. Please see details there. The pointer translation is calculated as ${X}_{vis} = {X}_{origin} + C * \alpha * random(-1, 1) * abs(V)$ and ${Y}_{vis} = {Y}_{origin} + C * \alpha * random(-1, 1) * abs(V)$.

Here, ${X}_{origin}$, ${Y}_{origin}$ are the original pointer position along the x-axis and y-axis. ${X}_{vis}$, ${Y}_{vis}$ is the translated pointer position along the x-axis and y-axis, which are visualized on screen. The translation amount is obtained as a product of $\alpha$, a random value sampled from the standard normal distribution, and the absolute value of the device’s velocity V. C is a constant. $\alpha$ is the coefficient unit, which means the “size of the visual oscillation.”

3 User Study 1

The first user study investigated whether visual oscillations modified the roughness judgment during finger-surface interactions. Ten participants, aged from 22 to 25 participated in the experiment. All participants were right-handed.

3.1 Experimental System

We manufactured texture plates by arranging glass particles (KENIS, Ltd) on acrylic plates following [6, 8]’s approach. Glass particles of different sizes were carefully attached to a flat acrylic plate with a double-sided sticky tape. In this study, we prepared nine texture plates with uniform surfaces, shown in Fig. 2. The diameters of the glass particles were 50, 75.5, 115, 163, 210, 425, 605, 850, and 1194 $\upmu$m.

Fig. 2.

Nine texture plates covered with glass particles. The diameters of the glass particles were different for different plates. The length and width of the plate were 100 mm.

Fig. 3.

(a) Environment of the user study. Top: Overview. Bottom Left: The sensor was attached to the participants’ index fingers. Bottom Right: Participants watched their finger movement through pointer on mirrored display. (b) Experimental window. There were left and right side.

Figure 3 (a) shows the experimental environment. One of the texture plates was placed about 30 cm in front of the participants. The texture plate was unseen directly from the participants, owing to a mirror. The position sensor of the tracker (Polhemus Fastrak) was attached to the participants’ index fingers.

The position of the index finger was updated at 120 Hz and sent to a personal computer (PC). Unity was used for visualizing the position of a finger, according to the sensing position at 60 Hz. The contact point was visualized as a black circle, and the circle diameter was three pixels, which was equivalent to 0.93 mm on display (ASUS VG278HE, 1920 $\times$ 1080, 81.589 dpi). From the viewpoint of the participant, the pointer appeared to be positioned as the same position as their index finger. The participants touched the texture plates with their index fingers, while watching two separate areas on the screen, and judged which area felt rougher (Fig. 3 (b)). The haptic information about both areas, acquired by touching the texture plate, was the same, since the glass particles’ diameters across the texture plate were the same. The width and height of the texture area displayed were adjusted to be the same as those of the real texture plate.

The pointer oscillated, and the position of the pointer was updated using the expression in section II. We conducted preliminary experiment to narrow down the $\alpha$ value where the pseudo-haptics were effective to some extent. According to the preliminary experiment, the user study used three different $\alpha$ values ($\alpha = 1, 1.5, 2$) to investigate the effect of size of $\alpha$ values. We refer to the three different $\alpha$ values as different “visual conditions”. C was configured so that the translation distance was 4.82 pixels (1.5 mm) when the pointer moved at 200 mm/s and $\alpha$ was 1. C was constant for all three $\alpha$.

We refer to the nine texture plates (from #1 to #9) as different “texture conditions”.

3.2 Task Design

This experiment used a within-participants design. The participants touched a texture plate under the mirror, watching the pointer in the left or right area on display. In One area, the pointer oscillated while in the other area, the pointer did not oscillated.

The assignment of the visual oscillation to the specific area (left or right) was randomly determined. After freely touching the plate, the participants were asked to report which area they felt to be rougher, by pushing the button. Since we wanted to reproduce the same situation as when touching a surface as usual, we did not define the time limit for each trial. We told participants to focus on the haptic roughness of the plates while watching the pointer on the screen and asked them to judge the roughness based on the haptic roughness instead of visual information. We allowed participants only to slide the pen on the surface and prohibited tapping or flicking gesture with pen. Also, participants were told that the texture plates were homogeneous.

There were nine texture conditions (from #1 to #9) and three visual conditions ($\alpha = 1, 1.5, 2$). For each condition, the participants repeated the trial four times. Thus, each participant conducted 108 trials ($= 9 \times 3 \times 4$). The texture and visual conditions were presented in the random order, and were counterbalanced across the participants.

3.3 Result and Discussion

First of all, to determine whether the participants’ answers were dissimilar from chance (50%), a chi-square goodness-of-fit test was performed on the frequency of the participants’ selections for two texture plates, against the null hypothesis that the two areas were equally selected. Figure 4 shows the number of times and a percentage a selection was made for the areas with visual oscillation for each visual and texture condition. Out of the 40 selections (10 participants multiplied by 4 times) under each condition for the question, the difference was observed to be statistically significant ($p\,<\,0.05$) for all conditions, except only one condition ($\alpha = 1$ and texture #1, $p\,=\,0.057$). The result of chi-square test and the biased selection to the areas with visual oscillation implied that the participants reported that the vibrotactile texture felt rougher compared with the case in which there was no visual oscillation.

Fig. 4.

Matrix along texture condition axis and visual condition axis on the frequency and percentage of selection for the areas with visual oscillation.

Next, we conducted an analysis using the generalized linear model with the binomial distribution. We compared the full model and each of the three reduced models which had variables of (1) only visual condition or (2) texture condition or (3) visual and texture condition but without interaction effect. Then, we conducted a likelihood ratio test (L.R.T) against the null hypothesis that each reduced model’s deviation and the full model’s deviation were same. We generated it using the chi-square test which is equivalent to L.R.T when performing logistic regression.

According to the result, the reduced model without texture condition was significantly different from the full model ($p\,<\,0.01$) and thus, the texture condition significantly affected the response. On the other hand, the reduced model without visual condition or interaction effect was not different from the full model ($p\,>\,0.05$). Thus, visual condition and interaction effect was not shown to affect the response.

The result of this user study suggests that visual oscillations may change the roughness judgment of real textures. However, under the experimental conditions, the participants may have answered only the presence or absence of the visual oscillations. Therefore, we conducted a user study 2 to evaluate the physical equivalent point from the viewpoint of tactile roughness, and indirectly confirmed the effect of visual oscillations.

4 User Study 2

The second user study investigated what size of visual oscillation is optimal to provide the desired, modified roughness. Six participants, aged from 22 to 25 participated in the experiment. All participants were right-handed.

4.1 Experimental System

The experimental environments except for texture plates’ arrangement and the experimental window were the same as user study 1. The same texture plates from #1 to #9 were used in this user study. The closest pairs of two texture plates in terms of roughness (e.g. [#1, #2]) were presented to participants at the same time. We refer to the eight pairs of texture plates (from [#1, #2] to [#8, #9]) as different “texture pair conditions”.

The visual oscillation was disabled for the rougher texture and was enabled for the smoother texture. Participants adjusted the size of visual oscillations $\alpha$ of smoother texture so that they felt the same roughness from both texture plate. The value of C was the same as previous user study.

Fig. 5.

(a) Participants touch two different rough texture plates. (b) Experimental window. There were left and right side, each corresponding to two plates.

4.2 Task Design

This experiment used a within-participants design. The participants touched two texture plates under the mirror. One of the two plates was rougher and another was smoother. Though the assignment of the left or right to the plates was randomly determined, which side was the target of adjustment as illustrated in the experimental window. The pointer visually oscillated when the smoother plate was touched, but the pointer did not oscillate when the rougher plate was touched. The participants were asked to adjust the size of the visual oscillations $\alpha$ of the smoother plate so that they felt the same roughness from both the texture plates. Participants pushed a button from “decrease (−0.04)”, “slight decrease (−0.02)”, “slight increase (+0.02)”, or “increase (+0.04)”. The initial value of $\alpha$ was set at 0. There was no time limit for adjustment and participants could push buttons repeatedly.

There were eight texture pair conditions. For each condition, the participants repeated the trial twice. The texture pair conditions were presented in the random order.

Fig. 6.

The adjusted visual oscillation size for smoother texture plate for equalizing roughness.

4.3 Result and Discussion

Figure 6 shows the results. The horizontal axis shows the smoother texture plate of texture pairs. The vertical axis shows the adjusted size of visual oscillation for the smoother texture of texture pairs.

If the visual oscillation does not affect the roughness, the adjustment value should be zero. To determine whether the adjustment value is different from zero, we conducted t-tests for each texture pair condition. We conducted the normality by the Shapiro-Wilk test for each texture pair condition in advance. The results of the t-test for all shows that the adjusted size of visual oscillation was significantly different from zero ($p < 0.05$) and adjusted values ranged from 0.8 to 1.5.

Next, we performed a one-way ANOVA with factors of texture pair condition on the adjusted size. We conducted a Mauchly’s test to check the sphericity criteria in advance of the ANOVA test. According to the ANOVA results, the texture pair condition did not significantly affect the adjusted size ($F(7, 84) = 2.12, p = 0.61$).

5 Conclusion

In this study, we applied the pseudo-haptic method using visual oscillations, to modify the subjective roughness judgment while scanning real textured plates with bare fingers. We manufactured various texture plates, with roughness ranging from several tens of micrometers to several millimeters. We conducted user study where users judged the roughness of plate under conditions with and without visual oscillations. The results showed that visual oscillations can modify the perceived roughness of real textured surfaces.