Keywords

1 Introduction

Among noncontact tactile presentation techniques using air jets [1], vortex rings [2], lasers [3], and ultrasound [4,5,6], the technique using ultrasound can reproduce several tactile sensations, including geometric features of surfaces. In this study, we assume the case where a finger moves along the surface of a virtual object and discuss the spatial resolution as the basic parameter. The base of this research is the pseudo-pressure presentation [7] recently demonstrated by Morisaki et al. Since static pressure sensation is generated by low-frequency lateral modulation (LM) [8], reproducing spatial unevenness by changing the presented pressure in synchronization with the finger movement is possible.

Multiple sensory signals are used for a human to perceive geometric features of a surface haptically. For example, when capturing a macroscopic shape whose curvature radius is larger than the finger size, the curvature is determined by synthesizing the trajectory and surface angle of the perceived finger using proprioceptors and cutaneous receptors, respectively, during the finger motion [9]. Meanwhile, high-frequency vibration induced by the hand’s stroking motion is the main factor when sensing the microscopic features of the surface. A fine surface structure whose representative length is smaller than the depth of the mechanical receptors in the skin or the spacing between the adjacent fingerprint ridges can induce overall vibration perceived using deep mechanoreceptors [10, 11].

This paper focuses on mesoscopic shapes, whose representative length ranges from a few millimeters to fingertip size. These shapes can be perceived using superficial mechanoreceptors as the spatial pattern. Here, we distinguish between these mesoscopic and microscopic sensations though the word roughness encompasses both. In noncontact tactile displays, it remains unclear to what extent ultrasound can present detailed mesoscopic shapes, since it has a relatively large wavelength of 8.5 mm in 40 kHz in the air, which is the typical frequency of the current midair haptics.

In related studies, Matsubayashi et al. [12, 13] proposed a method for displaying object shapes in a finger pad. However, these studies were limited to cases where the virtual objects vertically contact the skin. The surface roughness has already been considered in midair haptics studies [14, 15]. Nevertheless, since static pressure sensation cannot be reproduced, presenting static shapes is difficult except for some special situations [16].

This study considers two basic strategies for presenting mesoscopic shapes. One is the contact position change (CPC) method, in which we change the center of the contact area between the surface shape and fingertip synchronously with the fingertip position. The other is the contact strength change (CSC) method, in which we change the sum of the forces using the fingertip position. This CSC method is similar to the approach proposed by Howard et al. [17], however, both strategies focus on different scales. The CSC method focuses on mesoscopic shapes, whereas Howard et al.’s research focused on macroscopic shapes.

In real contact, both the position and intensity of the contact change, nonetheless, we intentionally present them separately to clarify the spatial resolution in both strategies.

Fig. 1.
figure 1

Overview of the shape presentation. (a): Contact position change (CPC) method and (b): Contact strength change (CSC) method. (c): Method to present two adjacent convex surfaces. (d): Illustration of the LM linear movement modulation.

Full size image

2 Methods

2.1 Overview of the Methods

Figure 1 shows the overview of the CPC and CSC methods. In the following experiments, the trajectory of the finger is on the horizontal plane, with no vertical motion. However, we assume that the finger moves on the uneven rigid body surface and estimate the contact point and contact force. In the calculation, we assume that the finger is a circular rigid body.

The CPC method, shown in Fig. 1(a), reproduces the surface irregularity by presenting the contact point estimated from the finger position, keeping the contact force constant. It is unclear whether the tactile stimuli reproduced using the CPC method expresses unevenness. However, since the tactile stimulus reproduced by the CPC method was perceived as “soft and easily deformable convex objects” in the preliminary experiment, it was considered a stimulus to express unevenness. When the skin traces the surface of a rigid object, the contact pressure changes according to the unevenness. Additionally, the height of the finger changes up and down. However, when the stimulus is presented by the CPC method, the presenting pressure is assumed to be constant and the trajectory of the finger is along the horizontal plane. In such a situation, the presented target surface is perceived as a very soft and easily deformable object.

Meanwhile, Fig. 1(b) shows a method for calculating the contact pressure using the CSC method. Here, the contact pressure is given proportional to the height of the contact point. In the case of contact with a real object, both the contact position and pressure change. In other words, by using CPC and CSC together appropriately, can reproduce slight irregularities on the surface of a rigid body. In this research, our scope excludes tactile reproduction, but we will examine how fine the unevenness pattern can be expressed by changing the position and intensity of the 40 kHz ultrasonic focus.

2.2 Contact Position Change Method

The focus position in the CPC method is given as follows. Assume the finger center is at \(x = x_f\) and the ultrasound focus position is expressed as \((x, y) = (x_p, y_p)\). As shown in Fig. 1(a), \((x_p, y_p)\) is obtained

$$\begin{aligned} x_p = \frac{r_c}{r_c+r_f}\cdot (x_f-x_c) + x_c,\quad y_p = r_f-\sqrt{r_f^2-(x_f-x_p)^2} \end{aligned}$$

from geometric considerations, where the radii of curvature of the finger and virtual object are \(r_f\) and \(r_c\), respectively, and the x-axis center of the virtual convex surface is \(x_c\). Here, we assume \(r_f = 7\) mm. Moreover, we set \(r_c\) at 5, 15, 25, and 35 mm, respectively, to investigate the virtual convex surfaces.

Two adjacent convex surfaces are presented in Fig. 1(c). The contact point is calculated as follows: First, we determine the centers of the circles \((x_1, y_c)\) and \((x_2, y_c)\), circle radius \(r_c\), and height h from the base plane. In this study, we fixed \(h = 5\) mm throughout the experiments. Also, two peaks are symmetrically arranged around \(x = 0\).

If the finger position \(x_f\) is negative or positive, it is considered the finger contacts the left or right circles, respectively. The contact point is only one and calculated as explained above, although, in real time, the finger might touch multiple points on the surface. The tactile stimuli reproduced using the CPC method in the preliminary experiment were perceived as two bumps.

2.3 Contact Strength Change Method

The CSC method is shown in Fig. 1(b). We assumed that the total contact force P is proportional to the height of the contact point, which is given as

$$\begin{aligned} P=\left\{ \begin{array}{ll} \frac{\theta '}{\pi /2}\cdot P_M &{} (\theta '\le \pi /2)\\ -\frac{\theta '}{\pi /2}\cdot P_M &{} (\theta ' > \pi /2) \end{array} \right. , \end{aligned}$$

where \(P_M\) is the Max Pressure, the maximum force strength of the device, and

$$\begin{aligned} \theta '=(\theta -\theta _1)\cdot \dfrac{\pi /2}{\pi /2-\theta _1}. \end{aligned}$$

where \(\theta \) is the angle indicating the contact point between the finger and virtual object, while \(\theta _1\) corresponds to the minimal force strength when the finger touches the plane.

In the CSC method, the calculated pressure is applied to the finger pad center. Similar to the CPC method, the same parameters were used in the CSC-method experiments as \(r_f = 7\) mm, \(h =\) 5 mm, and \(r_c\) at 5, 15, 25, and 35 mm, respectively.

2.4 Ultrasound Focus Point Presentation Method

In previous explanations, we simply used the term “contact pressure.” In midair ultrasound tactile presentation, however, one problem is that the stimulus is too weak to perceive when a time-constant radiation pressure is applied to a fixed point on the skin. To avoid this problem, we used a low-frequency LM so that the stimulus is perceived even when the finger is stationary.

Recent studies have shown that LM modulation at low frequencies 10 Hz produces pseudo-pressure sensation [7]. Thus, the finger can feel pressure sensations with distinct spatial localization. In this study, we used the linear movement LM as shown in Fig. 1(d). The focus is oscillated sinusoidally at f [Hz] along the finger at \(z=l\; \mathrm {sin}(2\pi f t)\), where z is the focus position along the finger. We set \(l =\) 1.2 cm and \(f = 15\) [Hz]. The number of points per cycle in the LM vibration was set to 50 to avoid unnecessary vibration.

Fig. 2.
figure 2

Overview of the experiment. (a) is the photograph of the system. (b) and (c) are the results of the experiments by (b): CPC method and (c): CSC method.

Full size image

3 Experiment

To clarify the threshold for discriminating convex surfaces, we developed a system and conducted experiments as follows.

3.1 System

Figure 2(a) shows the presentation system consisting of an infrared touch screen (ITS), GreenTouch, GT-IRTK156-1, 15.6 in., and an airborne ultrasound phased array (AUPA) [18].

ITS is a position sensor device that locates the two-dimensional position coordinates of a finger (or something else) using infrared transmitting and receiving tubes densely placed on the four sides of the screen. For this system, it was used to obtain the position coordinate in the moving direction of the finger (i.e., the x-axis direction in Fig. 2(a)). Meanwhile, AUPA is a device that creates an ultrasound focal point at an arbitrary three-dimensional position by controlling the amplitude and phase of each ultrasound transducer arranged in an array. Here, six AUPAs were used to form the focal point using the proposed methods described in Sect. 2.

The AUPA and ITS were synchronized by using virtual reality development environment (Unity 2020.3.13f1, produced by Unity Technologies) to put the ultrasound focus on the finger using the finger coordinates captured by the ITS. In addition, users were required to put their fingers on a linear rail to maintain a constant finger height.

3.2 Procedure

The following procedure was performed in the CPC and CSC methods. The procedure follows the declaration of Helsinki (2013), and participants provided written informed consent before the study.

First, before starting the experiment, we visually displayed the two convex surfaces with the video made by Unity. In the video, we prepared the assumed convex surfaces and the supposed finger was linked to the actual finger. Next, participants were asked to answer whether they felt two convex surfaces as one or two according to displayed curvatures and distances. During the experiment, participants listened to white noise to exclude the influence of the driving noise of AUPAs, and judged the convex number using their haptics without the video. In this convex surface presentation, the sensation was presented on the convex surfaces, and not on the plane surface. Also, the exploring speed of the finger was unspecified.

In the experiment, we prepared two convex surfaces with the equal heights and radii (\(h=5\) mm, \(r_c\) at 5, 15, 25, 35 mm, respectively), and obtained the discrimination threshold of the two convex surfaces using the 1-up, 1-down staircase method for each radius. In the beginning, the distance between the centers of both convex surfaces (henceforth, distance d, see Fig. 1(c)) was set to 0 mm. We asked participants to answer whether they felt one convex surface or two convex surfaces within 3-laps exploration. We changed the distance d by 2 mm according to their answers. If the participants felt one convex surface, d was increased until they could feel two convex surfaces. In contrast, if the participants felt two convex surfaces, d was decreased until they could feel one convex surface. We repeated this process seven times. The first reversal was ignored in subsequent analyses and the remaining six were averaged to obtain an estimate of the discrimination threshold. In this experiment, we had one measure per trial block. The presentation of the different radii was ordered randomly. The boundary of the two convex surfaces remained at the center of the rail (see Fig. 2(a)).

Participants consisted of ten males and two females, with a mean age (± standard deviation) of 25.3 ± 1.2 yrs. All participants had touched the ultrasound in the past, but this was the first time using their fingertips. Before the experiment, subjects were asked to touch the surfaces with the video for them to understand the procedure. Since there were no videos during the experiment, this preparation did not develop the ability to discriminate thresholds well.

3.3 Results

Figure 2(b) and (c) show the experimental results. The discrimination thresholds of each participant obtained by the staircase method were summarized in a box plot for each radius. Among the participants, one participant always felt one convex surface even when the distance was over 50 mm at all radii in the CSC method, and another participant felt both convex surfaces even though the distance was 0 mm \(r_c = 35\) mm of the CSC method. These discrimination thresholds were excluded from the box plots.

Comparing the CPC and CSC methods, we find that the discrimination threshold in the CPC method was generally smaller. In both methods, the smaller the radius of the convex surface, the smaller the discrimination threshold. The minimum threshold throughout the experiment was obtained at \(r_c=5\) mm in the CPC method, with a mean threshold of 8.0 mm and a standard deviation of 2.7 mm. Participants’ comments included: “The CSC method was more difficult to feel,” “Sometimes the number of convex surfaces was judged symbolically,” and “It was more difficult to understand the change from two convex surfaces to one convex surface than from one convex surface to two convex surfaces.” Although we did not specify the speed of the fingers, the measured speed ranged from about 20 to 100 [mm/s].

3.4 Discussion

Comparing the thresholds for both the CPC and CSC methods in each radius, the CPC method had a smaller discrimination threshold than the CSC method at all radii. Therefore, participants were identified the number of convex surfaces more easily using the CPC method. As the authors’ subjective comment, the CPC method provided a clearer sensation of the soft and fine bump profile especially when the height h was lower than about 5 mm (see Fig. 1(c)).

A reason for the lower effectiveness of the CSC method is the weakness of the ultrasonic force (\(1.6\;\;\mathrm {gf/cm}^2\) with 324 transducers [5]). Due to its small maximum strength, it was difficult to understand the difference when the intensity varied. This was also evident in the comments. In particular, when the radius was small in the CSC method, the stimulus occurred at a short horizontal distance, making it more difficult to judge. Therefore, some participants made logical judgments such as whether there was a stimulus difference, rather than report how they felt the curved surface, especially at \(r_c=5\) and 15 mm. Upon improving the presentation power in the future, the CSC method will become more effective and the threshold can be lowered.

However, the CPC method was effective as a method for transmitting mesoscopic features. In particular, the minimum threshold of \(8.0 \pm 2.7\) mm obtained at \(r_c = 5\) mm showed that we can discriminate the spatial pattern on a surface even if the distance d (see Fig. 1(c)) was within the diameter of a finger (about 14 mm) using the CPC method. As the authors’ subjective comment, “the fine profile was naturally perceived though it feels very soft.”

4 Conclusion

In this study, we discussed and evaluated the spatial resolution of mesoscopic shapes created using ultrasonic LM stimulation, in the situation where a finger moves along the surface of a virtual object. As an indicator of the spatial resolution, we used the minimum distance between two bumps by which we can distinguish them.

Both CPC and CSC methods were used to display the surface pattern. In each procedure, the contact position and the force strength were, respectively, controlled according to the finger position. The experimental results show that the CPC method more effectively conveyed mesoscopic features than the CSC method. In addition, it obtained the spatial resolution (minimum discriminable bump distance) of 8.0 mm with a standard deviation of 2.7 mm.