Cross-modal facilitation of visual and tactile motion

Robust and versatile perception of the world is augmented considerably when information from our five separate sensory systems is combined. Much recent evidence has demonstrated near-optimal integration across senses, but it remains unclear at what level the integration occurs, at a "sensory" or "decisional" level. Here we show that non-informative "pedestal" motion stimuli in one sensory modality (vision or touch) selectively lowers thresholds in the other, to the same degree as pedestals in the same modality: strong evidence for functionally important cross-sensory integration at early levels of sensory processing.

Robust and versatile perception of the world is augmented considerably when information from our five separate sensory systems is combined.Much recent evidence has demonstrated near-optimal integration across senses, but it remains unclear at what level the integration occurs, at a "sensory" or "decisional" level.Here we show that noninformative "pedestal" motion stimuli in one sensory modality (vision or touch) selectively lowers thresholds in the other, to the same degree as pedestals in the same modality: strong evidence for functionally important cross-sensory integration at early levels of sensory processing.
Many recent studies have shown that brain integrates information across the different senses in a statistically optimal manner [1][2][3] , but it is not always clear that the integration is functionally optimal.For example, for motion defined by both vision and audition, integration can be as strong for opposite as for matched directions of motion [4][5][6] .
We studied visual and tactile motion perception by measuring minimal speed increment motion thresholds over a wide range of base-speeds (from 0 to 25 cm/sec).The stimuli were physical wheels etched with a sinewave profile of 10 c/deg (Fig 1A).Subjects, seated at 57 cm, observed the front wheel with their index finger resting on the second (concealed from view).
We first measured visual and tactile speed increment as a function of base speed: subjects responded in two alternative forced choice which of two presentations seemed faster, yielding the discrimination thresholds reported by red symbols in Fig. 1 B. Both visual and tactile motion produced the characteristic "dipper function", where the thresholds initially decrease with base speed to a minimum at base speeds around 0.08 cm/sec, then proceeded to rise, roughly in proportion to base speed (Weber's law).The visual and tactile motion curves are very similar in form, both in absolute sensitivity and position of the dip, suggesting similar mechanisms may be at work.
We next repeated the study in a cross-modal condition, where the base-speed (pedestal) was presented in one modality, and the increment to be detected in the other.For example, in both intervals the tactile motion could be, say, 1 cm/s (hence non-informative), and only in the test interval was there the visual motion to be detected.Blue symbols of Fig. 1B show crossmodal visual and tactile thresholds over the range of base-speeds.Like the single-modality data, these data show a clear "dip", again at around 0.08 cm/s.However, the form of the curves differed from the others in that there was no rising limb with Weber-like behaviour.
To study the facilitation more closely, for five naïve observers we measured visual and tactile speed thresholds with and without 0.08 cm/s pedestals of the same or different modality.
Averaged results (normalised to base-threshold) are shown in Figs 2 A&B.For both vision and touch, pedestals of the same (red bars) or different (blue) modality both reduced thresholds considerably, by more than a factor of two.In both cases the average effect of the pedestal was as strong for the cross-modal as for the intra-modal condition.To examine whether this may be due to reducing temporal uncertainty, we substituted the cross-modal motion-pedestal for a sound of matched duration (defining precisely the temporal interval of motion): but the concurrent sounds had no effect on base thresholds (green bars).We also measured facilitation with cross-modal pedestals moving in the opposite direction to the tests, informing observes that this was the case: but again, even though this condition contained as much information as the same-direction condition (in that observers knew the direction was always opposite), this pedestal had no effect (cyan bars).Fig. 2C shows the individual data for all conditions, plotting the pedestal against the no-pedestal thresholds.Clearly all red and blue symbols (pedestal conditions of the same or crossed modality) lie under the equality line, showing facilitation, where the other two conditions at or above it, showing no facilitation.
The results of this study strongly suggest that visual and tactile motion share common mechanisms.Over a wide range of speeds, the sensitivity curves are very similar, both showing a dipper-like facilitation at around 0.08 cm/s.Most interestingly, at the speed of the dipper, there was cross-facilitation between the two modalities, vision facilitating touch and vice versa.
Discrimination functions in many domains follow a "dipper function", including contrast discrimination 7,8 , blur 9 , and visual motion 10 .Explanations for the dipper function generally involve non-linearities in the function that transduces physical events into neural signals.The function is assumed to accelerate positively at low speeds (effectively a thresholding mechanism), so a non-informative pedestal aids performance by shifting the operating range to the steepest part of the function.Other explanations involve spatiotemporal uncertainty 8 ; but the lack of facilitation by sound beeps speaks against this explanation here.That the pedestals of opposed direction did not facilitate threshold discrimination, even when observers knew the direction was inverted and "tried to take this into account", speaks against cognitive explanations: this condition contained as much information as the same-direction (in that the inversion was totally predictable), but this information could not be used to facilitate thresholds.
It seems for the facilitation to work, the motion must of in the same direction, and within narrow bounds of speed, pointing to neural combination.If we accept that the dipper function reflects a non-linear, threshold-like transduction, then it would seem that the neural combination occurs before this thresholding.This conclusion is far stronger than those to date 1, 2, Ernst, 20041, 2, Ernst, #2349 , in suggesting that the combination of sight and touch is not just statistically optimal, but occurs at a moderately low level of sensory processing, before thresholding is applied.
Several lines of evidence suggest that tactile and visual motion share much in common.
Tactile motion is subject to similar illusions observed with visual motion, including a motion flow after-effect, the "aperture problem" and the Ouchi illusion 11 .There is also evidence from fRMI studies that tactile motion activates similar cortical areas to those activated by vision, including area MT 12,13 .This would be an ideal candidate for the neural substance mediating the facilitation observed here.

Figure captions
Figure 1 Illustration of the stimuli and main results for intra-modal and cross modal facilitation.
A. Illustration of the stimuli: two physical wheels under independent computer control, etched with a sinewave profile of specific spatial frequency (10 c/deg).The front wheel is observed through a small aperture.B&C Incremental speed thresholds for visual (B) and tactile (C) motion, as a function of base speed, for two observers.Red circular symbols show uni-modal thresholds, where all signals are confined to the same modality, vision (B) or touch (C).
Observers were required to choose the interval containing the faster speed (V+ǻV cm/s) from base-speed (V cm/s).The blue symbols show thresholds for pedestals of different modality to the test, tactile (B) or visual (C).The pedestal moved at V cm/s in both trial intervals, the test increment at ǻV cm/s only during the test interval.Test speed varied from trial to trial, following the adaptive algorithm QUEST 14    Threshold with no pedestal (cm/sec) Threshold with pedestal (cm/sec) that homed in on threshold.Thresholds were calculated by fitting a raised cumulative Gaussian the data (150 trials per point), with error bars indicating 1SEM, calculated by bootstrap.No feedback was given in any condition.