Skip to main content
Log in

Visual and Verbal Narrative Comprehension in Children and Adolescents with Autism Spectrum Disorders: An ERP Study

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

We examined semantic processing in ASD children by presenting sentences with congruent or incongruent final words and visual narratives with congruent or incongruent final panels. An N400 effect to incongruent words appeared as compared to congruent ones, which was attenuated for the ASD children. We observed a negativity sustained to incongruous than congruous words, but only for the TD children. Incongruent panels evoked a greater fronto-central N400 amplitude than congruent panels in both groups. In addition, incongruent panels evoked a centro-parietal late positivity, only in controls. In conclusion, ASD children face processing deficits in both verbal and visual materials when integrating meaning across information, though such impairments may arise in different parts of the interpretive process, depending on the modality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Allen, M. H., Lincoln, A. J., & Kaufman, A. S. (1991). Sequential and simultaneous processing abilities of high-functioning autistic and language-impaired children. Journal of Autism and Developmental Disorders,21(4), 483–502.

    PubMed  Google Scholar 

  • American Psychiatric Association diagnostic and statistical manual of mental disorders: DSM-III-R (3rd ed.). Washington, DC: American Psychiatric Association (1987)

  • Amoruso, L., Gelormini, C., Aboitiz, F., González, M. A., Manes, F., Cardona, J., et al. (2013). N400 ERPs for actions: Building meaning in context. Frontiers in Human Neuroscience,7, 57.

    PubMed  PubMed Central  Google Scholar 

  • Bach, P., Gunter, T. C., Knoblich, G., Prinz, W., & Friederici, A. D. (2009). N400-like negativities in action perception reflect the activation of two components of an action representation. Social Neuroscience,4, 212–232.

    PubMed  Google Scholar 

  • Baron-Cohen, S., Leslie, A. M., & Frith, U. (1986). Mechanical, behavioural and intentional understanding of picture stories in autistic children. British Journal of Developmental Psychology,4(2), 113–125.

    Google Scholar 

  • Barrett, S. E., & Rugg, M. D. (1990). Event-related potentials and the semantic matching of pictures. Brain and Cognition,2, 201–212.

    Google Scholar 

  • Bentin, S., McCarthy, G., & Wood, C. C. (1985). Event-related potentials, lexical decision, and semantic priming. Electroencephalography & Clinical Neurophysiology,60, 353–355.

    Google Scholar 

  • Braeutigam, S., Swithenby, S. J., & Bailey, A. J. (2008). Contextual integration the unusual way: A magnetoencephalographic study of responses to semantic violation in individuals with autism spectrum disorders. European Journal of Neuroscience,27, 1026–1036.

    PubMed  Google Scholar 

  • Brandwein, A. B., Foxe, J. J., Russo, N. N., Altschuler, T. S., Gomes, H., & Molholm, S. (2011). The development of audiovisual multisensory integration across childhood and early adolescence: A high-density electrical mapping study. Cerebral Cortex (New York),21(5), 1042–1055.

    Google Scholar 

  • Brouwer, H., Fitz, H., & Hoeks, J. (2012). Getting real about semantic illusions: Rethinking the functional role of the P600 in language comprehension. Brain Research,1446, 127–143.

    PubMed  Google Scholar 

  • Camblin, C. C., Ledoux, K., Boudewy, M., Gordon, P. C., & Swaab, T. Y. (2007). Processing new and repeated names: Effects of coreference on repetition priming with speech and fast RSVP. Brain Research,1146, 172–184.

    PubMed  Google Scholar 

  • Coderre, E. (2019). Dismantling the “Visual Ease Assumption”: A review of visual narrative processing in clinical populations. Topics in Cognitive Science.

  • Coderre, E. L., Chernenok, M., Gordon, B., & Ledoux, K. (2017). Linguistic and non-linguistic semantic processing in individuals with autism spectrum disorders: An ERP study. Journal of autism and developmental disorders,47(3), 795–812.

    PubMed  Google Scholar 

  • Coderre, E. L., Cohn, N., Slipher, S. K., Chernenok, M., Ledoux, K., & Gordon, B. (2018). Brain and language visual and linguistic narrative comprehension in autism spectrum disorders: Neural evidence for modality-independent impairments. Brain and Language,186, 44–59.

    PubMed  Google Scholar 

  • Cohn, N. (2013). Visual narrative structure. Cognitive Science,37(3), 413–452.

    PubMed  Google Scholar 

  • Cohn, N. (2014). You’re a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science,38(7), 1317–1359.

    PubMed  PubMed Central  Google Scholar 

  • Cohn, N. (2019). Your brain on comics: A cognitive model of visual narrative comprehension. Topics in Cognitive Science, 1–35.

  • Cohn, N., & Kutas, M. (2015). Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. Neuropsychologia,77, 267–278.

    PubMed  PubMed Central  Google Scholar 

  • Cohn, N., & Maher, S. (2015). The notion of the motion: The neurocognition of motion lines in visual narratives. Brain Research,1601, 73–84.

    PubMed  PubMed Central  Google Scholar 

  • Cohn, N., & Kutas, M. (2017). What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive research: principles and implications,2 (1), 27.

    Google Scholar 

  • Cohn, N. & Paczynski M. (2019). The neurophysiology of event processing in language and visual events. In R. Truswell (Ed.), Handbook of event structure. (pp. 624–637). Oxford: Oxford University Press.

    Google Scholar 

  • Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P. J., & Kuperberg, G. R. (2012). (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive psychology,65(1), 1–38.

    PubMed  PubMed Central  Google Scholar 

  • Cohn, N., & Wittenberg, E. (2015). Action starring narratives and events: Structure and inference in visual narrative comprehension. Journal of Cognitive Psychology,27(7), 812–828.

    PubMed  Google Scholar 

  • Coulson, S., King, J. W., & Kutas, M. (1998). Expect the unexpected: Event-related brain response to morphosyntactic violations. Language and Cognitive Processes,13, 21–58.

    Google Scholar 

  • Dawson, M., Soulières, I., Ann Gernsbacher, M., & Mottron, L. (2007). The level and nature of autistic intelligence. Psychological Science,18, 657–662.

    PubMed  PubMed Central  Google Scholar 

  • de Mello, C. B., Argollo, N., Shayer, B. P. M., Abreu, N., Godinho, K., Durán, P., et al. (2011). Versão abreviada do WISC-III: correlação entre QI estimado e QI total em crianças brasileiras. Psicologia: Teoria e Pesquisa, 27(2), 149–155

  • Dunn, M. A., & Bates, J. C. (2005). Developmental change in neural processing of words by children with autism. Journal of Autism and Developmental Disorders,35, 361–376.

    PubMed  Google Scholar 

  • Dunn, M., & Vaughn, H., Jr. (1999). Electrophysiologic correlates of semantic classification in autistic and normal children. Development Neuropsychology,16, 79–99.

    Google Scholar 

  • Fishman, I., Yam, A., Bellugi, U., Lincoln, A., & Mills, D. (2011). Contrasting patterns of language associated brain activity in autism and Williams syndrome. Social Cognitive and Affective Neuroscience,6(5), 630–638.

    PubMed  Google Scholar 

  • Friedrich, M., & Friederici, A. D. (2004). N400-like semantic incongruity effect in 19-month-olds: Processing known words in picture contexts. Journal of Cognitive Neuroscience,16(8), 1465–1477.

    PubMed  Google Scholar 

  • Ganis, G., Kutas, M., & Sereno, M. I. (1996). The search for “common sense”: An electrophysiological study of the comprehension of words and pictures in reading. Journal of Cognitive Neuroscience,8(2), 89–106.

    PubMed  Google Scholar 

  • Goldstein, G., Beers, S. R., Siegel, D. J., & Minshew, N. J. (2001). A comparison of WAIS—R profiles in adults with high-functioning autism or differing subtypes of learning disability. Applied Neuropsychology,8(3), 148–154.

    PubMed  Google Scholar 

  • Gunter, T. C., Stowe, L. A., & Mulder, G. (1997). When syntax meets semantics. Psychophysiology,34, 660–676.

    PubMed  Google Scholar 

  • Hagoort, P., Brown, C., & Groothusen, J. (1993). The syntactic positive shift (SPS) as an ERP measure of syntactic processing. Lang. Cogn. Process,8, 439–483.

    Google Scholar 

  • Hahne, A., Eckstein, K., & Friederici, A. D. (2004). Brain signatures of syntactic and semantic processes during development. Journal of Cognitive Neuroscience,16(7), 1302–1318.

    PubMed  Google Scholar 

  • Hahne, A., & Friederici, A. D. (1999). Electrophysiological evidence for two steps in syntactic analysis: Early automatic and late controlled processes. Journal of Cognitive Neuroscience,11, 194–205.

    PubMed  Google Scholar 

  • Holland, S. K., Vannest, J., Mecoli, M., Jacola, L. M., Tillema, J.-M., Karunanayaka, P. R., et al. (2007). Functional MRI of language lateralization during development in children. International Journal of Audiology,46, 533–551.

    PubMed  PubMed Central  Google Scholar 

  • Johnels, J. Å., Hagberg, B., Gillberg, C., & Miniscalco, C. (2013). Narrative retelling in children with neurodevelopmental disorders: Is there a role for nonverbal temporal-sequencing skills? Scandinavian Journal of Psychology,54(5), 376–385.

    PubMed  Google Scholar 

  • Kamio, Y., & Toichi, M. (2000). Dual access to semantics in autism: Is pictorial access superior to verbal access? Journal of Child Psychology and Psychiatry,41, 859–867.

    PubMed  Google Scholar 

  • Kaufman, A. S., & Lichtenberger, E. O. (2006). Assessing adolescent and adult intelligence (3rd ed.). Hoboken: Wiley.

    Google Scholar 

  • Kujala, T., Lepistö, T., & Näätänen, R. (2013). The neural basis of aberrant speech in autism spectrum disorders. Neuroscience & Biobehavioral Reviews,37, 1–8.

    Google Scholar 

  • Kuperberg, G., Sitnikova, T., Caplan, D., and Holcomb, P. (2003). Electrophysiological distinctions in processing conceptual relation- ships within simple sentences. Cogn. Brain Res,17, 117–129.

    Google Scholar 

  • Kuperberg, G. (2007). Neural mechanisms of language comprehension: Challenges to syntax. Brain Research,1146, 23–49.

    PubMed  Google Scholar 

  • Kuperberg, G. R. (2013). The pro-active comprehender: What event-related potentials tell us about the dynamics of reading comprehension. In B. Miller, L. Cutting, & P. McCardle (Eds.), Unraveling the behavioral, neurobiological, and genetic components of reading comprehension (pp. 176–192). Baltimore: Paul Brookes Publishing.

    Google Scholar 

  • Kuperberg, G. R. (2016). Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events. Language, Cognition and Neuroscience,31(5), 602–616.

    PubMed  PubMed Central  Google Scholar 

  • Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology,62, 621–647.

    PubMed  PubMed Central  Google Scholar 

  • Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science,207, 203–208.

    PubMed  Google Scholar 

  • Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature,307, 161–163.

    PubMed  Google Scholar 

  • Lee, C. L., & Federmeier, K. D. (2008). To watch, to see, and to differ: An event-related potential study of concreteness effects as a function of word class and lexical ambiguity. Brain and Language,104, 145–158.

    PubMed  Google Scholar 

  • Lepistö, T., Kujala, A., Vanhala, R., Alku, P., Huotilainen, M., & Näätänen, R. (2005). The discrimination of and orienting to speech and non-speech sounds in children with autism. Brain Research,1066, 147–157.

    PubMed  Google Scholar 

  • Lord, C., Rutter, M., & Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders,24, 659–685.

    PubMed  Google Scholar 

  • Loschky, L., Hutson, J., Smith, M., Magliano, J., & Smith, T. (2017). Viewing Static Visual Narratives Through the Lens of the Scene Perception and Event Comprehension Theory (SPECT) "Viewing Static Visual Narratives Through the Lens of the Scene Perception and Event Comprehension Theory (SPECT)". In A. Dunst, J. Laubrock, & J. Wildfeuer (Eds.), Empirical Comics Research: Digital, Multimodal, and Cognitive Methods (pp. 217–238). London: Routledge.

    Google Scholar 

  • Manfredi, M., Adorni, R., & Proverbio, A. M. (2014). Why do we laugh at misfortunes? An electrophysiological exploration of comic situation processing. Neuropychologia,61, 324–334.

    Google Scholar 

  • Manfredi, M., Cohn, N., De Araújo Andreoli, M., & Boggio, P. S. (2018). Listening beyond seeing: Event-related potentials to audiovisual processing in visual narrative. Brain and Language,185, 1–8.

    PubMed  Google Scholar 

  • Manfredi, M., Cohn, N., & Kutas, M. (2017). When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative. Brain and Language,169, 28–38.

    PubMed  PubMed Central  Google Scholar 

  • Männel, C., & Friederici, A. D. (2011). Intonational phrase structure processing at different stages of syntax acquisition: ERP studies in 2-, 3-, and 6-year-old children. Developmental Science,14(4), 786–798.

    PubMed  Google Scholar 

  • McCleery, J. P., Ceponiene, R., Burner, K. M., Townsend, J., Kinnear, M., & Schreibman, L. (2010). Neural correlates of verbal and nonverbal semantic integration in children with autism spectrum disorders. Journal of Child Psychology and Psychiatry and Allied Disciplines,51(3), 277–286.

    Google Scholar 

  • McPherson, W. B., & Holcomb, P. J. (1999). An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology,36(1), 53–65.

    PubMed  Google Scholar 

  • Nakazawa, J. (2005). Development of manga (comic book) literacy in children. In D. W. Shwalb, J. Nakazawa, & B. J. Shwalb (Eds.), Applied developmental psychology: Theory, practice, and research from Japan (pp. 23–42). Greenwich, CT: Information Age Publishing.

    Google Scholar 

  • Nakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In N. Cohn (Ed.), The visual narrative reader (pp. 157–184). London: Bloomsbury.

    Google Scholar 

  • O’Connor, K. (2012). Auditory processing in autism spectrum disorder: A review. Neuroscience and Biobehavioral Reviews,36(2), 836–854.

    PubMed  Google Scholar 

  • Olivares, E. I., Iglesias, J., & Bobes, M. A. (1999). Searching for face-specific long latency ERPs: A topographic study of effects associated with mismatching features. Cognitive Brain Research,7, 343–356.

    PubMed  Google Scholar 

  • Osterhout, L., & Holcomb, P. (1992). Event-related brain potentials elicited by syntactic anomaly. Journal of Memory and Language,31, 785–806.

    Google Scholar 

  • Pijnacker, J., Geurts, B., van Lambalgen, M., Buitelaar, J., & Hagoort, P. (2010). Exceptions and anomalies: An ERP study on context sensitivity in autism. Neuropsychologia,48(10), 2940–2951.

    PubMed  Google Scholar 

  • Proverbio, A. M., & Riva, F. (2009). RP and N400 ERP components reflect semantic violations in visual processing of human actions. Neuroscience Letters,459, 142–146.

    PubMed  Google Scholar 

  • Quante, L., Bölte, J., & Zwitserlood, P. (2018). Dissociating predictability, plausibility and possibility of sentence continuations in reading: Evidence from late-positivity ERPs. PeerJ 1–24.

  • Ribeiro, T. C., Valasek, C. A., Minati, L., & Boggio, P. S. (2013). Altered semantic integration in autism beyond language: A cross-modal event-related potentials study. NeuroReport,24(8), 414–418.

    PubMed  Google Scholar 

  • Sahyoun, C. P., Soulières, I., Belliveau, J. W., Mottron, L., & Mody, M. (2009). Cognitive differences in pictorial reasoning between high-functioning autism and Asperger’s syndrome. Journal of Autism and Developmental Disorders,39(7), 1014–1023.

    PubMed  PubMed Central  Google Scholar 

  • Schneider, J. M., Abel, A. D., Ogiela, D. A., Middleton, A. E., & Maguire, M. J. (2016). Developmental differences in beta and theta power during sentence processing. Developmental Cognitive Neuroscience,19, 19–30.

    PubMed  PubMed Central  Google Scholar 

  • Sitnikova, T., Holcomb, P. J., Kiyonaga, K. A., & Kuperberg, G. R. (2008). Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. Journal of Cognitive Neuroscience,20, 1–21.

    Google Scholar 

  • Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003). Semantic integration in videos of real-world events: An electrophysiological investigation. Psychophysiology,40(1), 160–164.

    PubMed  Google Scholar 

  • Soulières, I., Dawson, M., Samson, F., Barbeau, E. B., Sahyoun, C. P., Strangman, G. E., et al. (2009). Enhanced visual processing contributes to matrix reasoning in autism. Human Brain Mapping,30, 4082–4107.

    PubMed  PubMed Central  Google Scholar 

  • Tanner, D., & Van Hell, J. G. (2014). ERPs reveal individual differences in morphosyntactic processing. Neuropsychologia,56, 289–301.

    PubMed  Google Scholar 

  • Van Berkum, J. J., Hagoort, P., & Brown, C. M. (1999). Semantic integration in sen- tences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience,11(6), 657–671.

    PubMed  Google Scholar 

  • Van Berkum, J. J., Zwitserlood, P., Hagoort, P., & Brown, C. M. (2003). When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. Cognitive Brain Research,17(3), 701–718.

    PubMed  Google Scholar 

  • Van Petten, C., & Kutas, M. (1991). Influences of semantic and syntactic context on open and closed class words. Memory and Cognition,19, 95–112.

    PubMed  Google Scholar 

  • Van Petten, C., & Luka, B. J. (2006). Neural localization of semantic context effects in electromagnetic and hemodynamic studies. Brain and Language,97, 279–293.

    PubMed  Google Scholar 

  • Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs, and ERP components. International Journal of Psychophysiology,83, 176–190.

    PubMed  Google Scholar 

  • Wechsler, D. (1981a). WAIS-R manual: Wechsler adult intelligence scale-revised. Cleveland, OH: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (1981b). Wechsler Adult Intelligence Scale—Revised. Cleveland, OH: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (1992). Wechsler Intelligence Scale for Children (3rd edn rev). NewYork: Psychological Corporation.

    Google Scholar 

  • Wechsler, D. (1999). Wechsler Intelligence Scale for Children. New York: Psychological Corporation.

    Google Scholar 

  • West, W. C., & Holcomb, P. J. (2002). Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research,13, 363–375.

    PubMed  Google Scholar 

  • Zalla, T., Labruyère, N., Clément, A., & Georgieff, N. (2010). Predicting ensuing actions in children and adolescents with autism spectrum disorders. Experimental Brain Research,201, 809–819.

    PubMed  Google Scholar 

Download references

Acknowledgments

Mirella Manfredi was supported by a FAPESP post-doctoral research grant (2015/00553-5). Paulo S. Boggio is supported by a CNPq research grant (311641/2015-6).

Author information

Authors and Affiliations

Authors

Contributions

M.M. conceived and designed the study, analyzed data and wrote the paper. M.M, N.C. and P.S.B. interpreted the data. E.F. and P. S. M.developed stimuli, gathered behavioral pilot data and EEG data.

Corresponding author

Correspondence to Mirella Manfredi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Manfredi, M., Cohn, N., Sanchez Mello, P. et al. Visual and Verbal Narrative Comprehension in Children and Adolescents with Autism Spectrum Disorders: An ERP Study. J Autism Dev Disord 50, 2658–2672 (2020). https://doi.org/10.1007/s10803-020-04374-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-020-04374-x

Keywords

Navigation