An extended multisensory temporal binding window in autism spectrum disorders
- First Online:
- Cite this article as:
- Foss-Feig, J.H., Kwakye, L.D., Cascio, C.J. et al. Exp Brain Res (2010) 203: 381. doi:10.1007/s00221-010-2240-4
- 2.7k Downloads
Autism spectrum disorders (ASD) form a continuum of neurodevelopmental disorders, characterized by deficits in communication and reciprocal social interaction, as well as by repetitive behaviors and restricted interests. Sensory disturbances are also frequently reported in clinical and autobiographical accounts. However, surprisingly few empirical studies have characterized the fundamental features of sensory and multisensory processing in ASD. The current study is structured to test for potential differences in multisensory temporal function in ASD by making use of a temporally dependent, low-level multisensory illusion. In this illusion, the presentation of a single flash of light accompanied by multiple sounds often results in the illusory perception of multiple flashes. By systematically varying the temporal structure of the audiovisual stimuli, a “temporal window” within which these stimuli are likely to be bound into a single perceptual entity can be defined. The results of this study revealed that children with ASD report the flash-beep illusion over an extended range of stimulus onset asynchronies relative to children with typical development, suggesting that children with ASD have altered multisensory temporal function. These findings provide valuable new insights into our understanding of sensory processing in ASD and may hold promise for the development of more sensitive diagnostic measures and improved remediation strategies.
KeywordsAutismMultisensoryTemporal bindingAudiovisualSensory processingCross-modal integration
Autism spectrum disorders (ASD) comprise a continuum of neurodevelopmental disorders typically characterized by a triad of symptoms that include deficits in social reciprocity and communication skills, and repetitive behaviors and restricted interests (APA 2000). In addition, reports of altered sensory processing abound in autobiographical, caregiver and clinical reports, detailing a host of sensory aversions, sensitivities, and fascinations in individuals with ASD (Williams 1994; Kientz and Dunn 1997; O’Neill and Jones 1997; Sigman and Capps 1997; Dawson and Watling 2000; Grandin 2000; Talay-Ongan and Wood 2000; Watling et al. 2001; Wing and Potter 2002; Rogers et al. 2003; Baranek et al. 2006; Leekam et al. 2007). Indeed, reports of sensory disturbances date back to Kanner’s original description of autism (Kanner 1943).
Several recent empirical studies have further highlighted changes in sensory processes in individuals with ASD. Interestingly, some of these studies have shown superior visual, auditory, and somatosensory perceptual discrimination in individuals with ASD relative to control subjects (Mottron et al. 2006; O’Riordan and Passetti 2006; Cascio et al. 2008). For example, pitch discrimination (i.e., the ability to differentiate two tones of similar frequency) is enhanced in individuals with autism (Bonnel et al. 2003). Other studies suggest that these enhanced perceptual abilities are limited to fairly simple stimuli and that disrupted performance characterizes responses as stimuli become more complex (Bertone et al. 2005; Minshew and Hobson 2008). In addition to these differences in sensory processing within individual sensory systems, there is emerging evidence that alterations in the integration of information across the different senses (i.e., multisensory integration; see Iarocci and McDonald 2006) may exist in individuals with ASD, though strong empirical support for this is lacking. Such multisensory integration characterizes much of everyday experience, as we are continually bombarded with stimuli from multiple sensory modalities and must make judgments as to which stimuli belong together and which are unrelated. One of the most important and compelling forms of multisensory integration lies in the speech perception domain, where we use both auditory and visual cues to enhance the intelligibility of the speech signal. Consistent with disrupted multisensory processing in ASD, a number of studies have highlighted deficits in speech processing associated with autism (Williams et al. 2004; Smith and Bennetto 2007).
In an effort to account for the observed dissociation between performance on simple and complex perceptual tasks in individuals with ASD, it has been theorized that the critical deficit may lie in the temporal synchronization among both local and distributed neural networks (Brock et al. 2002). These networks can show strong patterns of entrainment in response to a given sensory stimulus (i.e., a focus of activation in one area is soon followed in a strongly time-locked fashion by a focus in a second connected brain area), and this temporal synchronization among brain regions is likely to be critically important in the binding of multisensory stimuli into unified perceptual constructs (Senkowski et al. 2008). In support of deficits in temporal processing in the context of ASD, several studies have shown differences in various aspects of sensory temporal function, including duration and rate processing (Tecchio et al. 2003; Szelag et al. 2004; Gomot et al. 2006). When examined across domains (i.e., simple vs. complex tasks), specific deficits have been shown in complex (i.e., speech-related) processes (Bebko et al. 2006; Magnée et al. 2008).
A recent study sought to directly examine multisensory processing of simple stimuli in ASD using the sound-induced double-flash (“flash-beep”) illusion (Van der Smagt et al. 2007). In this illusion, pairing a single visual stimulus (i.e., flash) with several auditory stimuli (i.e., beeps) often results in the perception of two or more flashes in typical adults (Shams et al. 2000). Van der Smagt and colleagues found that individuals with ASD are also susceptible to this illusion, suggesting intact multisensory binding mechanisms. However, the temporal dependence of the flash-beep illusion was not explored in this study, despite previous findings that the illusion is critically dependent on the temporal structure of the visual and auditory cues (i.e., as the time between the auditory cues and the single flash increases, the perception of illusory flashes weakens) (Shams et al. 2002). Given previous findings of impaired temporal processing in ASD, we hypothesized that changes in the temporal structure of the visual and auditory cues in the flash-beep task might reveal differences in the temporal “binding window” for multisensory stimuli in individuals with ASD. The concept of this binding window is an intuitive one, and reflects the fact that there are brain mechanisms that strive to unify two events (or stimuli) separated by a short interval of time.
12.60 ± 2.6
12.09 ± 2.2
105.10 ± 17.6
109.41 ± 12.5
109.80 ± 18.3
103.41 ± 7.32
Full Scale IQn.s.
108.45 ± 18.7
107.29 ± 9.3
Social Communication Questionnaire**
19.84 ± 8.1
2.71 ± 2.3
Parents of all participants gave informed consent and all children in both groups gave assent prior to participation in any component of this study. All children received compensation for their participation at each visit. All recruitment and experimental procedures were approved by the Vanderbilt University Institutional Review Board.
Participants sat in a light- and sound-attenuated room and wore headphones through which auditory stimuli were presented. They indicated their responses to the visual task stimuli, presented on a computer monitor, through button presses on a response box. Visual stimuli were presented as white flashes against a black background on a high-refresh rate PC monitor (NEC Multisync FE992, 22 in. screen; 150 Hz refresh rate; 640 × 480 pixel resolution). Auditory stimuli were presented via noise-canceling extra-aural headphones (Philips SBC HN110) to both ears (peak sound level 96 dB SPL). Stimulus presentation was controlled using E-Prime (Psychology Software Tools Inc., Pittsburgh, PA, USA). Responses (i.e., accuracy and response time) were recorded via a Serial Response box (Psychology Software Tools Inc., Pittsburgh, PA, USA).
Participants were monitored by the experimenter, using closed-circuit CCD video cameras, to ensure that they were engaged in the tasks. Eye gaze was not monitored, but participants were instructed to fixate on a central cross that preceded all stimulus presentations. On the rare occasions when a participant was not on-task, a variety of strategies were implemented to increase engagement (e.g., reminders to stay on task, additional breaks, parent in the testing room, etc.). Participants were allowed to take breaks as necessary to increase compliance and maintain effort, motivation, and on-task behavior. All participants completed the experimental task within a single session.
The mean percentage of trials on which two flashes were reported at each one-flash/two-beep SOA condition was calculated separately for each individual.
Differences in the proportion of trials on which an illusory flash was reported (i.e., the participant indicated seeing two flashes when only one was presented) were examined between groups using a repeated measures ANOVA with SOA as a within-subjects factor and group as a between-subjects factor. Independent sample t tests at each SOA condition were also conducted to determine specific SOAs showing group differences. Performance differences on the one-flash/one-beep control condition were examined in a similar manner to test for any group-specific response bias.
Determination of temporal windows
In an effort to provide a unitary measure of the processing differences between the two groups, the temporal “window” for the flash-beep illusion was defined as the contiguous span of consecutive one-flash/two-beep SOAs throughout which the mean percentage of two flashes reported was significantly greater than the mean percentage of two flashes reported on the one-flash/one-beep control (non-illusory) condition. To examine this temporal window of multisensory integration in children with ASD and TD, paired-sample t tests comparing the proportion of trials on which two flashes were reported for each one-flash/two-beep SOA condition to the one-flash/one-beep control condition were conducted separately for the ASD and TD groups. Corrections for multiple comparisons were not conducted because the method of analysis described above was planned a priori. Family-wise error was limited in the determination of the temporal window by requiring continuous significant differences from the one-flash/one-beep condition across the entire window.
The proportion of trials on which participants perceived two flashes was determined at each of the SOA conditions that manipulated the temporal structure of the single flash and two beeps (i.e., one-flash/two-beep conditions). Higher proportions of reported perception of two flashes indicated a greater strength of illusion. Between-group comparisons in the proportion of trials on which two flashes were reported were conducted for each of the one-flash/two-beep SOA conditions as well as for the one-flash/one-beep condition, which served as a control condition against which to measure response bias. On the one-flash/one-beep (non-illusory) condition, children in both groups did not always report a single flash, indicating that there was some degree of response bias across all children. In both groups, the percentage of trials on which two flashes were reported was significantly different from zero [ASD group (mean = 15%; SD = 20%): t(20) = 3.47, p = 0.002; TD group (mean = 8%; SD = 13%): t(16) = 2.60, p = 0.02)]. Most importantly, these values, and thus the assumed response bias, did not differ between groups (t(36) = 1.20, p = 0.24).
For the one-flash/two-beep conditions, a repeated measures ANOVA with SOA as a within-subjects factor and group as a between-subjects factor was conducted. The main effect of SOA was significant (F(16, 576) = 33.55, p < 0.001), confirming the observed relationship between the temporal disparity between the auditory and visual cues (i.e., SOA) and probability of integration. The main effect of group was also significant (F(1, 36) = 4.33, p < 0.05), indicating that the two groups differed overall in their likelihood of reporting an illusory second flash. The interaction between group and SOA was not significant (F(16, 576) = 0.85, p = 0.63), indicating that the global relation between temporal disparity and probability of integration is similar for children with ASD and TD.
Determination of temporal windows
An additional analysis was structured in order to define differences in the temporal window of multisensory integration between children with ASD and TD. In children with TD, significant increases in the proportion of trials on which two flashes were reported (above the one-flash/one-beep baseline) were seen at the following one-flash/two-beep SOAs: −150, −100, −50, −25, +25, +50, +100, and +150 ms (all p’s < 0.005). In comparison, in children with ASD, significant increases in the proportion of trials on which two flashes were reported were seen at the following SOAs: −500, −300, −200, −150, −100, −50, −25, +25, +50, +100, +150, +200, and +300 ms (all p’s < 0.05). These findings suggest an approximate doubling in the size of the temporal binding window in children with ASD, in that the contiguous span of SOAs at which the illusion was observed is approximately 300 ms in TD (i.e., from −150 to +150 ms) and approximately 600 ms in ASD (i.e., from −300 to +300 ms) (Fig. 2). Further validating the significance of these findings is the observation that the windows defined for each group show continuous significance at all SOAs within the window. Similarly, the between-group comparisons show continual significance at the SOAs outside the TD temporal window but inside the ASD temporal window (i.e., −200, +200, −300, and +300 ms).
The results of the current study suggest that children with ASD have an extended temporal window within which they bind together multisensory stimuli, as evidenced by their heightened propensity to report the flash-beep illusion over an extended range of SOAs between the component visual and auditory stimuli. The lack of significant differences between groups for the control trials (e.g., one-flash/one-beep) indicates that the difference in the temporal window size between groups is unlikely to be due to differences in response bias. While the tendency to report one flash in the two-flashes/zero-beeps condition across both children with ASD and TD suggests that visual temporal acuity may be lower in children relative to adults (Irwin et al. 1985; Hautus et al. 2003; Smith et al. 2006), the lack of differences between groups suggests that developmental differences in visual temporal acuity do not play a role in the highlighted perceptual differences between children with and without ASD. Although our results are in accord with a previous study showing intact integration of low-level visual and auditory stimuli in individuals with ASD (i.e., that integration of multisensory information does occur) (Van der Smagt et al. 2007), we have refined our understanding by showing for the first time alterations in the temporal constraints within which audiovisual stimuli are bound in children with ASD. The finding of intact integrative processes is in contrast to prior studies that have reported a decreased ability for individuals with ASD to integrate information across multiple modalities (Williams et al. 2004; Smith and Bennetto 2007). However, these studies focused on audiovisual speech stimuli, which are rich in social and contextual information and typically are also associated with affective demands. The processing of these communication signals may itself be altered in ASD, making it difficult to parse apart alterations in basic sensory function. Consistent with this interpretation is work that has reported that children with ASD perform comparably to children with TD on multisensory tasks involving non-speech stimuli but disparately on multisensory tasks involving speech stimuli (Mongillo et al. 2008). The current study confirms that individuals with ASD are able to integrate simple, non-linguistic audiovisual information. However, our results also highlight a striking difference in the integration of low-level multisensory stimuli, specifically in the temporal constraints within which auditory stimuli can influence visual perceptions in generating a compelling illusion.
There are several possible neurophysiological mechanisms for the enlarged temporal binding window seen in children with ASD, which fit within the conceptual framework of previously proposed neurally based models. Brock et al. (2002) have posited that a core neurological cause of autism may be rooted in disruptions in temporal processing. According to this theory, perceptual binding is a result of strongly correlated activity among a network of interconnected brain regions, and alterations in these patterns of correlation in ASD result in concomitant reductions in binding. The current study suggests that rather than these networks being completely decoupled in ASD, the time constants between brain regions may instead be altered in such a way so as to continue to support binding, but over an atypically large set of temporal intervals. A second proposed neural mechanism for ASD is founded on a decreased signal-to-noise ratio in neural encoding (Rubenstein and Merzenich 2003). In this view, under typical conditions, a briefly presented unisensory (e.g., auditory) stimulus typically results in a discrete neural response time-locked to the presentation of the stimulus. In contrast, the same stimulus presented to an individual with autism may result in a response whose neural signature is less clearly time-locked to the stimulus event. Extending this theory into the multisensory domain, it can be envisioned that increased temporal variability in the unisensory responses could necessitate a compensatory enlargement in the time interval over which multisensory stimuli can influence one another. Future studies will focus on devising methods for distinguishing between these and other potential neural mechanisms for the extended temporal binding window in ASD.
The current study must also be interpreted in the context of recent work that has focused on theories of multisensory function grounded in concepts of causal inference. In these models, the brain makes probabilistic judgments about the relatedness of two stimuli in an effort to build a coherent perceptual representation (Ernst and Banks 2002; Alais and Burr 2004; Kording et al. 2007). One important factor in these probability judgments is the temporal structure of the combined stimuli, and alterations in temporal processing would be expected to change the relative weighting of the inputs contributing to this cue combination process. Encouragingly, these multisensory binding processes (and the neural processes that underlie them), which develop during the course of early life as a function of sensory experience, remain plastic into adulthood (e.g., Powers et al. 2009). Such work provides hope that deficits in multisensory function in individuals with ASD may be ameliorated through perceptual training approaches, a strategy that has been proposed for other clinical conditions in which multisensory temporal function appears to be disrupted (i.e., dyslexia, see Hairston et al. 2005).
Autism spectrum disorders are extremely heterogeneous, and our task and study design limited us to evaluating children with relatively intact intellectual abilities (i.e., IQ score above 70). Thus, our findings may not generalize to lower functioning individuals with ASD and a concomitant intellectual disability. Although the task employs low-level stimuli and simple behavioral responses (hence offering promise for extending it to more impaired participants), continued adaptation and streamlining of this experimental design for use with a broader sample of children with ASD will be the focus of future research.
The extended temporal window for multisensory integration described in the current study is likely to have far-reaching consequences for children with ASD. At a very basic level, an alteration in the characteristics of the incoming sensory stream will have profound implications for all brain regions and processes “upstream” of the impacted (multi)sensory domain, since the integrity of the sensory signaling will have been altered or compromised. Differences in the processing and integration of sensory stimuli for individuals with ASD could underlie the atypical responses to sensory stimuli so frequently reported in the autism clinical literature. For instance, if integration is occurring over an extended temporal window, it could cause difficulty with responding to input from a specific modality if there is concurrent input from other modalities. Difficulties identifying the source modality of information, as have been reported in ASD (Cesaroni and Garber 1991), could also be explained by altered multisensory temporal function. In addition, numerous activities of daily life are dependent on the ability of the nervous system to precisely match stimuli from multiple modalities. For example, the dynamic auditory and visual stimuli involved in any social interchange (e.g., subtle changes in facial expression, tone of voice, body language) must all be integrated sequentially and seamlessly with precise temporal accuracy for the interaction to be successful. Thus, misalignment or inappropriate integration of basic sensory information would likely negatively impact individual interactions by changing the information content and, with such altered experiences repeated over time, would be expected to impair complex social abilities such as empathy and reciprocity as well as endow social interaction with confusing and irrelevant associations. The results in the current study could also be relevant to others’ findings of reduced integration in more complex multisensory (e.g., speech) stimuli, though future research is necessary to elucidate the role an expanded temporal window for binding low-level sensory stimuli plays in impaired integration of higher order cross-modal input.
In conclusion, this study represents an important first step in our understanding of the temporal processing of multisensory stimuli in ASD. Further research is needed to fully characterize the extent of these multisensory processing changes in ASD, to elucidate their neural substrates, and to relate these findings to the core deficits in ASD. It is anticipated that this line of investigation will ultimately contribute to a broader understanding of this disorder and lead to improved diagnostic instruments and more targeted interventions.
This work was supported by a Marino Autism Research Institute Discovery Grant (PI: MTW); National Institute of Child Health and Development T32 HD07226 predoctoral fellowship for JHF; Meharry-Vanderbilt Alliance Training Grant for LDK; Vanderbilt Kennedy Center.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.