Keywords

1 Introduction

Haptic perception is inherently active. To achieve high levels of perceptual performance and to behave most efficiently during haptic exploration, humans adapt their exploratory behavior to the present objects and task [3,4,5]. Without prior information about object properties, this motor adaptation is enabled by closed-loop processes that are based on sensory information, gathered and integrated over the sequential exploration process [2, 6, 7]. However, in many situations humans have prior information about the objects they are going to interact with. The current study investigates how they integrate prior information in a subsequent haptic exploration process.

Prior information can arise e.g. from recent interactions with similar objects, from semantic [8] or visual information [9]. Most of the times humans look at objects before they interact with them; hereby they can gather information about properties as shape, size or orientation. They can use this to adapt manual motor behavior already at initial contact [10]. A number of previous studies have examined the use of prior information on weight in grasping and lifting behavior [11] or on softness in exploratory behavior [1, 7]. For softness discrimination, it was shown that humans explore stimuli with higher initial peak forces when they expect harder as compared to softer stimuli. Here, we study the use of prior visual information on orientation in texture exploration.

Haptic texture exploration is a crucial subset of tools that humans employ during object discrimination and it is typically performed by moving the fingertips laterally across an object’s surface several times [5]. Thus, movements produce small patterns of vibrations, i.e. temporal cues that enable humans to discriminate between fine textures by their microgeometry [12]. To investigate texture exploration, grating patterns are particularly useful: their surfaces consist of periodically repeating grooves which define a clear orientation. For patterns with a defined orientation, efficiency of the intake of temporal cues can be maximized by moving orthogonal to the grating direction. Lezkan and Drewing [2] showed that indeed exploratory direction in grating discrimination is optimized accordingly based on sensory information, gathered over the course of an exploration: In their experiment participants had to discriminate gratings by their spatial frequency. In each trial they explored two types of textures: One single grating texture (defined orientation) and a texture made of two combined gratings (no defined orientation). Participants adapted their final movement directions orthogonal to the texture orientation when exploring the single grating stimulus. In another experiment this adaption was shown to be beneficial in terms of perceptual precision as it led to lower just noticeable differences (JNDs) for spatial frequency discrimination. We used a similar method and examined whether prior visual information on grating orientation evokes the same adaption of movement direction as in [2], but already for initial movements. Moreover, according to ideal observer models, humans should weigh less noisy prior information higher than noisy prior information [8]. Hence, we hypothesized that humans show more adaptation behavior when receiving prior information with higher as compared to lower quality, and presented information with different qualities.

2 Methodology

2.1 Participants

16 right-handed students from Giessen University participated (10 female, mean age: 23 y., range: 18‒27). All had normal or corrected-to-normal vision, reported no tenosynovitis in the past and no motor or cutaneous impairments. They had a 2-point discrimination threshold <4 mm on the right fingertip, were naïve to the purpose of the experiment, provided written informed consent and received financial compensation (8€/h). The experiment was approved by the local ethics committee LEK FB06 and conducted in accordance with 2013 Declaration of Helsinki, except for preregistration.

2.2 Stimuli and Setup

Participants sat at a custom-made visuo-haptic workbench (Fig. 1a), consisting of a PHANToM 1.5A haptic force feedback device (spatial resolution: 0.03 mm, temporal resolution: 1000 Hz), a force sensor to collect data of the executed finger force (682 Hz, resolution: 0.05 N), and a 24ʺ computer screen (120 Hz, 1600 × 900 pixel). They looked at the screen through a mirror and stereo glasses (Nvidia 3D Vision 2, viewing distance 40 cm). The visual setup displayed 3D scenes aligned with the haptic workspace and prevented that participants saw their hand. In the scenes, grating stimuli were displayed as light grey cylindrical discs on a dark green checkerboard. At the same positions two real stimuli were placed side-by-side on the force sensor in front of the participant. A small sphere (8 mm diameter) represented the participants’ finger position in the scene but disappeared as soon as they touched the stimulus. A chinrest stabilized the head position. The index finger was connected to the PHANToM via a spherical magnet fixed at the fingernail, allowing to move the finger in all axes with the maximum amount of freedom in a 38 × 27 × 20 cm3 workspace. The adapter left the fingertip free for bare-finger exploration. Devices were connected to a PC where C++-based custom software controlled the experiment and processed the data. Acoustic noises were masked with passive noise-cancelling headphones and white noise. As haptic stimuli we used four 3D-printed grating stimuli (Fig. 1b; printer: Stratasys Objet 30 Pro, resolution of 600 × 600 × 1600 dpi). We printed 4 mm high (z-axis) grating discs with a texture diameter of 90.7 mm (100.7 mm including the border). A 10 × 5 mm grip helped the experimenter arranging the stimuli before each trial. All stimuli consisted of a groove pattern following a sine-wave function in height z(x):

$$ z = \frac{1}{2}A\sin \frac{2\pi x}{P} + \frac{1}{2}A $$
(1)

Textures were defined to have the same amplitude A of 0.3 mm, differing only in their period P. The stimuli had a period of 1.524 mm, 2.032 mm, 2.540 mm, and 3.048 mm. Stimuli with adjacent periods were compared (=three pairs). Every pair was presented in 6 possible orientations relative to the observer (15°, 45°, 75°, 105°, 135° and 165°; for 0° ridges would be parallel to the body, see Fig. 1b). For prior visual information, visual representations of the grating stimuli were displayed with a texture made of 100 dark grey stripes (8 × 1 mm) on their top side (Fig. 1c). The textures indicated the orientation of the following haptic gratings and were displayed before exploration. Quality of prior information was manipulated by varying the percentage of stripes following the same orientation as the gratings’ ridges while the remaining ones were randomly oriented. Pilot studies (N = 16 overall) had shown that participants perceived the orientation of such stimuli excellently when 100% to 50% of the stripes were identically oriented, but variance in answers systematically increased with lower percentages. We hence presented visual stimuli with percentages of 50% or lower.

2.3 Procedure and Design

We varied the quality of prior visual information on stimulus orientation in five steps (50%, 35%, 25%, 15%, and 0% [=no information]). Stimuli were presented in 6 possible orientations (15°, 45°, 75°, 105°, 135° and 165°). We presented three different pairs of stimuli with adjacent half-periods in both possible left-right assignments. In both assignments participants started exploration equally often with the left and the right stimulus. They conducted a 2AFC discrimination task: On each trial they explored the two stimuli and had to decide which one had the higher spatial frequency.

Overall, each participant conducted 360 experimental trials (5 qualities × 6 orientations × 3 pairs × 2 stimulus location × 2 start location)in randomized order plus 8 initial practice trials. We implemented a break of 3 min every 60 trials; the experiment took about 3.5 h in a single session. During each single trial, prior information was displayed for 2500 ms before participants were allowed to explore the haptic stimuli. Exploration was initiated by a beep sound. One of the two visual stimulus representations discs was colored in a bright yellow to indicate where participants should start exploration. Participants were instructed to use the typical movement scheme for this, i.e. stroking laterally over the surface [11]. They were free to switch between stimuli as often as desired. After the exploration they indicated which stimulus they had perceived to be of higher frequency by pressing a virtual button above it. During the whole experiment white noise was presented through the headphones to mask any exploration sounds containing information about spatial frequency. After the experiment, participants filled out a questionnaire in order to check whether they had paid attention to the (not explained) visual prior information and intentionally used it for adaption purposes. They were not informed about the relation between prior information and the gratings’ orientation beforehand.

Fig. 1.
figure 1

a, visuo-haptic workbench b, haptic stimulus in orientation 165° c, examples of visual prior information with qualities 0%, 25% and 50% (from left to right) in orientation 135°.

Full size image

2.4 Data Analysis

Raw data of individual observers is available at 10.5281/zenodo.5718623. Movement directions of the first, middle and last stroke in the exploration of each stimulus pair were analyzed. In case of an even number of total strokes, the later of the two possible ones was defined as the middle stroke. An algorithm segregated single strokes from movement data when the participants’ finger was touching the stimulus area with at least 0.1 N of force for >200 ms. We detected strokes as continuous movements either from one texture border to another or between two movement turns, being extracted by zero crossings in the 1st order derivatives of the x- or y-position over time. We were particularly interested in the initial movement direction as it can be assumed to be hardly affected by sensory feedback and is thus an indicator of the use of prior information. Movement direction was determined as the orientation of the line connecting start and end point (x and z coordinates) of each stroke. Note that movement directions were processed as axial data and thus ranged only between 0°‒180°. We aligned all stimulus orientations with 0° in order to collapse data over trials. We calculated circular histograms of initial movement directions (bin size: 10°) separately for each visual quality condition. Each histogram displays how many times participants moved in a specific direction. Movement directions were analyzed using circular statistics: We used V-tests on the distributions of individual average movement directions per visual quality condition and separately for the first, middle and last stroke in order to test whether the distribution is not uniform (=all directions are equally likely) but rather has a specified mean direction of 90° (=optimal adaption behavior). Significant test statistics imply a deviation from uniformity in the suspected direction. Additionally, we compared proportions of close-to-orthogonal strokes (90° ± 15°) between quality conditions across participants by using a repeated measures ANOVA with the within-participant factor Visual Quality. Likewise, the mean resultant vector length \(\overline{R}\) of each participants’ initial movement directions for each visual quality condition entered a repeated measures ANOVA to compare the variance of movement directions between quality conditions. \(\overline{R}\) is the vector length of the mean direction \(\overline{\theta }\) of a circular distribution. It varies between zero and one; an \(\overline{R}\) near one implies that there is little variation in movement directions and the data is concentrated around the mean direction \(\overline{\theta }\).

3 Results

On average, participants spent 5.5 s (SD = 3.6) per trial on the stimuli, performed 6.6 strokes (SD = 4.8), switched 1.7 times (SD = 1.1) between them and gave 91.1% correct responses (SD = 2.8%). A repeated measures ANOVA with the within-participant factor Visual Quality did not show differences in the arcsine-transformed percentages of correct responses, F(4,60) = 0.18, p = .951, \(\eta_p^2 = .03\).

We plotted angular distributions of movement directions of all initial strokes separately for each visual quality condition (Fig. 2a‒e). V-tests were performed on the distributions of individual average movement directions per quality condition, and separately for the first, middle and last strokes (Bonferroni-corrected α-levels at 0.0033 for 15 tests). For initial strokes, movement distributions of the 25%, 35% and 50% quality condition deviated significantly from uniformity, 25%: V = 11.71, p < .001, 35%: V = 9.54, p < .001, 50%: V = 9.15, p < .001, suggesting optimization of movement behavior. Distributions of the 0% and 15% quality condition did not deviate significantly, 0%: V = 7.26, p = .005, 15%: V = 6.56, p = 0.012. For individual average movement directions of the middle and last strokes, distributions of all quality conditions deviated significantly from uniformity (all p < .001, all V > 11.37). Proportions of movements close-to-orthogonal to the textures’ surface 90° (±15°) among all initial movements across participants (Fig. 2f) entered an ANOVA with the within-participant factor Visual Quality. As expected, proportion of orthogonal movements was higher with higher visual qualities, F(4, 60) = 4.39, p = .036, \(\eta_p^2 = .23\) (Greenhouse-Geisser adjusted [13]), confirmed by a linear trend, F(1,15) = 6.11, p = .026, \(\eta_p^2 = .29\). For middle and last movements, the proportions of strokes close-to-orthogonal to the textures’ surface did not differ between quality conditions, F(4,60) = 2.331, p = .114, \(\eta_p^2 = .14\) (Greenhouse-Geisser adjusted) and F(4,60) = 1.52, p = .208, \(\eta_p^2 = .092\).

Fig. 2.
figure 2

a‒e, Initial movement direction histograms for each quality condition including all participant data. Textures were aligned to a 0° orientation. Note, possible movement directions varied only between 0–180° and were mirrored on the lower part of each figure. Orange lines indicate mean directions, black bars in the center of each circle represent resultant vector lengths. Black circle segments mark the 95% confidence interval of the mean direction. f, average percentages of orthogonal movements and standard errors for each quality condition and movement type.

Full size image

For the initial movements, which are at focus here, also the individual mean resulting vector lengths, being indices of variance in movement direction, entered an ANOVA with the within-participant factor Visual Quality. Vectors were longer with higher visual qualities, i.e. variability of the initial movements’ directions decreased with higher visual quality, F(4, 60) = 4.81, p = .03, \(\eta_p^2 = .243\) (Greenhouse-Geisser adjusted), confirmed by a linear trend, F(1,15) = 6.16, p = .025, \(\eta_p^2 = .30\) (0%: .11 ± .09[M ± SD], 15%: .21 ± .18, 25%: .23 ± .27, 35%: .28 ± .27, 50%: .29 ± .28). Questionnaire data showed that only two participants were aware that prior information indicated the orientation of the upcoming haptic stimulus and intentionally used it to adapt their exploration behavior.

4 Discussion

The aim of this study was to investigate the influence of prior visual information that indicates a grating orientation on the direction of (initial) exploratory movements in a grating discrimination task. Previous findings had shown that, without prior information, participants adapt the direction of their exploratory movement to be orthogonal to the orientation of textures in later strokes of the exploration, i.e., based on the sensory information previously gathered [2]. The results here clearly show that with prior information participants adapt already the direction of their initial stroke to the stimulus orientation. Moreover, we found a positive relationship between the quality of prior visual information and the extent of adaptation, corroborating our interpretation of an adaptation that is based on prior information. In particular, we found that the percentage of initial orthogonal strokes increases, and the variance of movement direction decreases with increasing quality of prior information. Importantly, individuals do not necessarily have to be aware of this process in order to show said behavior. In line with that, Zöller et al. [1] even observed that explicit signals, in contrast to implicit predictive signals, did not induce adaption of force in a softness discrimination task. Overall, we conclude that prior information on texture orientation is used to hone exploratory behavior in grating discrimination—via a pre-attentive, possibly lower-level process.

It has previously been demonstrated that the execution of movements orthogonal to a gratings’ orientation leads to a decrease in the JNDs between two textures’ spatial periods [2]. Thus, higher perceptual precision, i.e., a higher percentage of correct responses, with higher qualities of prior information–mediated by a higher extent of adaption towards orthogonal with higher qualities—would have been a plausible observation in the current context. We, however, did not find differences in the percentage correct between quality conditions. One reason for this lack of effect could be that participants were free in their explorations and able to compensate for unfavorable initial strokes, e.g. by performing extra strokes later on. This is represented in the descriptive data but not in a statistically significant manner. Another reason could be task difficulty, which was relatively low in the present study as stimuli differed by approximately 2 Wb fractions from each other in their spatial period [14]. Differences in perceptual precision between quality conditions might be less pronounced due to ceiling effects, but should be present when task demands are higher. Future investigations might manipulate task difficulty by using lower differences between stimuli. High task demands might affect the use of prior information and thus the adaption of exploration behavior differently than low demands. A manipulation of task difficulty could even reveal an interaction effect: Higher task demands might increase adaption behavior compared to low demands especially for lower information qualities, as behavioral optimization is needed more under difficult circumstances and to achieve optimization, even low quality prior information might be used. Furthermore, the effect of sensory signals on movement adjustment during softness discrimination has been shown before to partly depend on reward-induced task relevance, as the signature of force tuning gets stronger with higher motivation [7]. This might be the case for prior information too, leading to maximized adaption behavior when prior information is given and motivation is highest.

In conclusion, it became evident that prior visual information is given substantial weight to in the context of exploratory movement planning; and of which the individual does not necessarily have to be aware. Whether this resembles unconscious learning or rather inherent processing in humans is an interesting question for a future study. Eventually, findings on human use of prior information can be applied in the field of e.g. neurorobotics to enhance the performance of AI-based object recognition [15]. The implementation of optimized exploration processes could facilitate a robotic system’s information gathering, leading to a more efficient and more accurate decision-making.