In this work, we used a biomimetic tactile fingertip to sense mid-air haptics, to develop a method for testing the output of a haptic display. We measured the response of the tactile sensor to a focal point of pressure generated by an ultrasonic haptic array as well as two haptic shapes, each generated by Amplitude Modulation (AM) and Spatiotemporal Modulation (STM).
3.1 Sensing Mid-Air Haptics on a Two-Dimensional Grid
The experiments showed that the biomimetic tactile fingertip used in this study, the TacTip, is able to sense the mid-air haptic stimuli produced by the ultrasonic phased array using our developed method (Fig. 2). The focal points of pressure generated by the ultrasound caused the skin-like surface to deform, expanding the areas of the cells in the Voronoi tessellation, allowing us to identify the location and intensity of contact. Voronoi tessellation was a valuable tool in transducing a third dimension in the data, which allowed us to visualize the sensations produced by the ultrasonic array.
Our analysis methods enables us to produce detailed visualizations of the mid-air haptic sensations (Fig. 3), allowing us to distinguish between different patterns produced by the ultrasonic array. Additionally, the variation in the strength of the focal point can be clearly seen. The visual representation of the focal point shows that it creates a localized region of increased displacement (Fig. 2, lower right panel). The point is much stronger in the center, and then decreases in intensity as you move radially outwards. This is similar for the other shapes; the center path of the shape has increased intensity, which decreases as you move away (Fig. 3).
The visualizations produced by our method allow us to compare the shapes generated by the ultrasonic array using different modulation techniques. We see that the tactile sensor is able to distinguish the four focal points that make up the amplitude modulated line (Fig. 3, top left panel). A user of the ultrasonic array would not distinguish the points as the distance between them is small
, and so it creates the illusion of a continuous line. The sensor can discriminate between the points because our analysis method is measuring the deformation of the tactile sensor’s surface, which would correspond to the deformation of the user’s skin rather than their perception of the sensation. On the other hand, the spatiotemporally modulated line (Fig. 3, lower left panel) is felt as a continuous line by the sensor. The focal point used to generate the line is moved along its path at very small increments. The distance between the points in this case is too small to be distinguished by the sensor, making the output more similar to how a user would sense the stimulus.
3.2 Sensing a Focal Point in Three-Dimensional Space
In the previous section, we presented our results when sensing various shapes over a two-dimensional grid. In this section, we extend our method to sense mid-air haptic stimuli in three dimensions. When a person interacts with the haptic array to sense the shapes it generates, they naturally move their hand around the display surface, which includes moving their hands up and down as they process the sensations they feel. Thus, understanding how the generated sensations vary with height is important to determine whether the desired effect is being produced and to check that there are no undesired artefacts. We repeated the data collection process described earlier for the point stimulus at different heights over the array, at 10 mm increments. This results in a three-dimensional grid on which we applied our presented analysis method.
This experiment allows us to see how the shape is sensed over a three-dimensional surface, looking at how the shape varies by height. The focal point is generated by the array at a specific height in space; however, there are still sensations at other points due to the interaction of the ultrasonic waves
. The point stimulus is sensed by the tactile fingertip as an elongated spheroid, with a localized region of increased intensity (Fig. 4). It appears the lower the intensity of the stimulus felt, the more elongated it is. As the sensor moves away from the center height, the stimulus becomes fainter.