Temporal sequence discrimination within and across senses: do we really hear what we see?

Abstract

Previous evidence suggests that people “hear” visual stimuli when encoding temporal information. This suggestion is based on the observation that auditory distractor information can strongly affect discrimination performance for visual temporal sequences. The present study aimed to replicate and extend this finding by investigating sequence discrimination within and across the two modalities. In two experimental series, participants judged whether two subsequently presented temporal sequences, a standard sequence followed by a comparison sequence, were identical or not. In Experimental Series A, irrelevant distractor information was presented simultaneously with the standard sequence. In Series B, the distraction appeared in the retention interval between the standard sequence and the comparison sequence. The results showed that auditory distraction impaired performance irrespective of whether the target sequences were auditory or visual, whereas visual distraction only impaired the discrimination of visual target sequences. Furthermore, auditory distraction was always at least as effective as visual distraction, irrespective of standard modality. Generally, discrimination performance was much better for auditory than for visual sequences. Overall, the present results are consistent with the idea that people code visual temporal information in the auditory modality. Moreover, the present study also suggests that such cross-modal interference effects should be interpreted cautiously with respect to their underlying timing mechanism because of the basic differences in temporal sensitivity between the two modalities.