Keywords

1 Introduction

When we touch natural surfaces, we are extremely good at distinguishing different materials (e.g. silk from satin) despite the complexity of the patterns of stimulation elicited by tactile exploration [1].

Touching a surface causes patterns of vibrations on our skin which are sensed by the mechanoreceptors embedded in the skin. Softness and temperature information is also available. The vibratory signals play an important role for perceiving natural textures [2,3,4]. Perceptual representations based only on vibratory signals acquired indirectly with a tool are remarkably similar to representations obtained with bare hand exploration [4]. These vibratory signals are highly dependent on exploration movements (e.g. speed) and on the local properties of textures.

In order to understand the relationship between the vibratory signals elicited by free exploration of natural textures and how we perceive these textures while we touch them, we built a database of vibratory patterns recorded while human participants freely explored a large set of natural textures with a steel tool tip. After the exploration of each signal, participants provided judgments about their perceptual attributes (e.g. roughness or friction) and rated how much the explored material feels like each of the seven material categories used in the experiment (wood, plastic, fabric, paper, metal, stone and animal).

Here we describe the properties of the vibratory signals we recorded in relation to the perceptual judgements provided by human participants. The spectral power P relates to the temporal frequency f according to a power law (\(P = \frac{1}{f^{s}}\)). We found that vibratory signals can be approximated by a single parameter s in the temporal frequency domain. The same relationship characterizes the spatial frequency spectrum of natural visual textures (e.g. [5]). Crucially, this parameter s varies systematically between material categories such that it can be used to predict physical category labels (e.g. metal vs. plastic) better than chance. Classification performance improves when classifying perceptually assigned labels, suggesting a relationship between perception of material categories and the spectral statistic described by s. In fact, s correlates with human judgements of some of the perceptual attributes participants rated.

In our previous research, we showed that haptic perceptual representations emerge by efficient encoding of vibratory signals [4]. These representations resemble the responses of the RA and PC afferents, tuned to lower temporal frequencies (peak 50 Hz) and higher (250 Hz), respectively [6]. Here we show that s correlates with the ratio of the PC to the RA responses; i.e. the higher the contribution of high temporal frequencies as compared to lower frequencies, the higher s. Thus, s provides a concise measure of the temporal frequency composition of the vibratory signals elicited by the exploration of natural images, provides information about material categories and perceptual attributes, and can be computed by comparing the responses of mechanoreceptors.

2 Methods

2.1 Participants

Eleven students volunteered to participate in the experiment; all were naïve to the purpose of the experiment and were reimbursed for their participation. The study was approved by the local ethics committee LEK FB06 at Giessen University and was in line with the declaration of Helsinki from 2008. Written informed consent was obtained from each participant.

2.2 Stimuli

Our natural textures consisted of 81 different material samples which were glued on wooden pieces (14 \(\times \) 14 cm, Fig. 1). These materials samples are the same used by [7]. They belong to seven material categories: plastic, paper, fabric, fur and leather, stone, metal and wood, and were chosen to represent the large variety of materials we encounter in everyday life.

Fig. 1.
figure 1

Photographs of all material samples. Rows indicate different material categories: plastic, paper, fabric, fur and leather, stone, metal and wood, from top to bottom.

Full size image

2.3 Procedure and Apparatus

Participants set at a table looking at a computer monitor elevated by a support. Material samples were positioned by the experimenter in front of them through a hole in the support so that they could easily be touched but no visual information was available. The sound from the exploration of materials was covered by earplugs and white noise presented via headphones.

Participants freely explored the 81 surfaces with a 3D printed pen containing a steel tip at its end and a mounted accelerometer (ADXL345). This way we could record the vibrations elicited by the interaction between the steel tip and material samples. Each participant explored each material once.

The onset of the white noise signalled that they could begin the exploration. They were instructed to slide the pen over the material’s surface. After the exploration of each material, participants rated how much the explored material felt like each of the seven material categories (paper, fabric, animal, stone, plastic, wood, metal) on a scale from “very different" to “very similar”. Then, they rated how much the material could be described by each of seven opposing adjective pairs. We used a subset of the descriptors used by [7]: rough vs. smooth, hard vs. soft, orderly vs. chaotic, warm vs. cold, elastic vs. not elastic, high friction vs. slippery, textured/patterned vs. homogeneous/uniform. The experimental software was written with Psychopy [8].

2.4 Vibratory Signals

For each material, 10 s of recording were acquired at 3200 Hz temporal resolution. We started the recording after 2 s of exploration and stopped 2 s before the exploration was terminated (i.e. participants explored 14 s each sample), to prevent that signals are affected by contact onset and offset. We filtered out frequencies 10 Hz as they may be ascribed to exploratory hand movements [9,10,11,12]. We cleaned the signals by removing frequencies 800 Hz, which are not relevant for perception of texture properties of materials [1,2,3, 13, 14] and may be caused by measuring noise.

2.5 Analysis

We approximated the relationship between temporal frequency f and the amplitude power P with the following function \(P = \frac{1}{f^{s}}\) In a log-log space, this equals to the following linear relationship \(P = {-s}{f}\). We determined s by fitting a line in log-log space. From now on we refer to s as the slope (of the line). We assessed how well this linear relationship can approximate the power spectrum by computing \(R^{2}\), i.e. the proportion of variance explained by the linear fit. Slope was computed for each material and each participant separately, i.e. for each exploration trial. Since signals only included power at frequencies 10 Hz and 800 Hz, the linear fits were performed within this interval.

We used slope to classify the material categories by means of a linear classifier. For classification, we averaged slope across participants yielding one average slope per material. To prevent over-fitting, we iteratively left out one material and trained the classifier on the remaining slopes, then computed performance on the left-out material. We used bootstrap analysis to test whether classification performance was higher than chance: we repeated the classification analysis 5000 times shuffling the category labels every time. Thus we computed the distribution of classification accuracy under the null hypothesis of chance-level classification. The 95% confidence interval was computed by reading out the \(2.5^{th}\) and the \(97.5^{th}\) percentiles of the distribution. The empirical chance level corresponded to the mean of that distribution. We repeated the classification analysis based on perceptual labels. To determine perceptual category labels, first we assigned to each material the highest rated category by each participant, then we chose the most frequent category, i.e. the one chosen by the majority of participants.

To relate slope to the responses of the mechanoreceptors, we estimated the responses of the RA and PC afferents using the TouchSim toolbox [15].

3 Results

Figure 2 shows one example of vibratory signal per material category. For all categories the linear model seems to provide a good approximation of the spectral profile.

Fig. 2.
figure 2

Spectral profiles of example materials for each of the seven categories. Natural logarithm of the amplitude spectrum on the y-axis, as a function of the natural logarithm of temporal frequency (x-axis). Each diagram represents the spectral profile of one example material, explored by one participant in one experimental trial.

Full size image

On average, the linear fits could explain 43% of the variance of the spectral profiles, indicating that almost half of the variability in such high dimensional signals (32000 dimensions) can be approximated by two parameters (slope and intercept). We used a linear classification analysis to determine whether slope variations across different vibratory patterns provide information about material properties. Based on slope we could classify material categories better than chance (classification performance = 33.3%; empirical chance level = 14.8 %, with [6.17 23.46] 95% Confidence interval). Performance increased when we repeated the classification analysis based on perceptually assigned categorical labels (classification performance = 38.27%; empirical chance level = 16.67%, with [6.17 27.16] 95% Confidence interval). This means that human misclassifications could be explained by differences in slope, suggesting that perception is at least in part based on slope. To explore this possibility, we investigated the relationship between slope and human ratings of perceptual attributes (Fig. 3).

Fig. 3.
figure 3

(a–g) Relationship between human ratings of perceptual attributes and slope. Z-transformed perceptual ratings averaged across participants, on the y-axis. Slope on the x-axis. (h) Relationship between activity of mechanoreceptors and slope. The differential activity of the PC and RA afferents is represented by the natural logarithm of the ratio of the activity of the PC to the RA afferent (y-axis). Slope on the x-axis. Colors in the legend in the top left panel indicate different material categories. Dashed lines indicate regression lines.

Full size image

Correlation analyses show that shallower slope is significantly associated with low roughness, elasticity, fiction and with high hardness judgments (r = −0.31, r = −0.45, r = −0.31, r = 0.35; all p-values < \(\alpha \), with (\(\alpha \) = 0.00714), according to Bonferroni correction for seven post-hoc comparisons).

RA and PC afferents seem crucial for perception of natural textures [3, 4]. RA respond to relative low temporal frequencies, whereas PC to higher frequencies. Therefore, the differential activity of these afferents provides information about the composition of the power spectrum of the tactile input. We quantified this information as the natural logarithm of the ratio between the activity of the RA and the PC afferents (PC/RA). For each vibratory signal we computed the RA and PC responses using the TouchSim toolbox [15], and related this quantity to slope (Fig. 3).

The ratio PC/RA correlates with slope (r = 0.48, p < 0.00001), indicating that they employ a similar way to summarize the power spectrum of vibratory signals elicited by exploration of natural materials.

4 Discussion

We recorded the vibratory signals elicited by free exploration of a large number of natural materials. Each exploration is associated to perceptual judgments. Other databases of vibratory signals have been published. Vibratory signals were measured with the same system we used at controlled speed and free exploration [9, 10, 12]. To our knowledge, our database is the only one to present perceptual judgments, effectively ruling out the influence of visual and audio information. Furthermore, we focused on everyday life materials rather than following industrial conventions for sampling and naming materials. Our database with vibratory signals and perceptual judgements (ViPer) is publicly available (https://github.com/matteo-toscani-24-01-1985/ViPer). Other attempts have been made to relate perceptual judgments to vibratory signals elicited by natural materials. However, the vibration signals were elicited by exploratory motion by robots [16, 17].

We showed that vibratory signals can be approximated by a single parameter representing the slope of the linear relationship between spectral power and temporal frequency in log-log space in the frequency domain. This parameter can be used to classify material categories. Classification performance improves when classifying perceptual labels, suggesting that slope can account for human misclassifications. The bond between slope and perception is strengthened by the correlations with human judgements of perceptual attributes like roughness, hardness, elasticity and friction.

The same linear relationship characterizes the spatial frequency spectrum of natural visual textures (e.g. [5]). This may indicate scale invariance of the visual world. Spatially rescaling an image by a factor of \(\alpha \) implies rescaling the corresponding frequency domain axes by a factor of \(\frac{1}{\alpha }\). A power spectrum that falls as a power law will retain its shape under this transformation, i.e. would show scale invariance. We speculate that the slope we used to describe vibratory signals may exhibit time invariance, i.e. would change minimally for explorations at different speed. Hence, slope could be estimated based on the differential activity of the RA and PC afferents (as suggested by Fig. 3h) and used to perceive material properties despite different velocities of exploratory movements, i.e. to achieve speed invariance [18]. However, we did not measure the movement speed, therefore we cannot test how much it affects slope.

We previously showed that haptic perceptual representations emerge by efficient encoding of vibratory signals [4]. Such representations can be described within a space whose dimensions resemble the activity of RA and PC afferents. In this space, the representations of different materials tend to lay on a line along which it is possible to distinguish between different material categories, i.e. materials can be distinguished based on the differential activity of the RA and PC afferents. As slope is able to capture the differential activity of the RA and PC afferents, we speculate that slope may be a prominent feature of the compressed representation we previously discovered.

Our analyses showed that a simple statistical property of the Fourier spectrum is able to capture nearly half of the variability within the vibratory signals elicited by a large number of natural textures. This property systematically differs between material categories and correlates with perceptual judgments. Our results may represent a significant step ahead for tactile rendering, just like in vision pink noise is used for synthesizing naturalistic textures (e.g. [19]).