Visual and Verbal Narrative Comprehension in Children and Adolescents with Autism Spectrum Disorders: An ERP Study

Abstract

We examined semantic processing in ASD children by presenting sentences with congruent or incongruent final words and visual narratives with congruent or incongruent final panels. An N400 effect to incongruent words appeared as compared to congruent ones, which was attenuated for the ASD children. We observed a negativity sustained to incongruous than congruous words, but only for the TD children. Incongruent panels evoked a greater fronto-central N400 amplitude than congruent panels in both groups. In addition, incongruent panels evoked a centro-parietal late positivity, only in controls. In conclusion, ASD children face processing deficits in both verbal and visual materials when integrating meaning across information, though such impairments may arise in different parts of the interpretive process, depending on the modality.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Allen, M. H., Lincoln, A. J., & Kaufman, A. S. (1991). Sequential and simultaneous processing abilities of high-functioning autistic and language-impaired children. Journal of Autism and Developmental Disorders,21(4), 483–502.

    PubMed  Google Scholar 

  2. American Psychiatric Association diagnostic and statistical manual of mental disorders: DSM-III-R (3rd ed.). Washington, DC: American Psychiatric Association (1987)

  3. Amoruso, L., Gelormini, C., Aboitiz, F., González, M. A., Manes, F., Cardona, J., et al. (2013). N400 ERPs for actions: Building meaning in context. Frontiers in Human Neuroscience,7, 57.

    PubMed  PubMed Central  Google Scholar 

  4. Bach, P., Gunter, T. C., Knoblich, G., Prinz, W., & Friederici, A. D. (2009). N400-like negativities in action perception reflect the activation of two components of an action representation. Social Neuroscience,4, 212–232.

    PubMed  Google Scholar 

  5. Baron-Cohen, S., Leslie, A. M., & Frith, U. (1986). Mechanical, behavioural and intentional understanding of picture stories in autistic children. British Journal of Developmental Psychology,4(2), 113–125.

    Google Scholar 

  6. Barrett, S. E., & Rugg, M. D. (1990). Event-related potentials and the semantic matching of pictures. Brain and Cognition,2, 201–212.

    Google Scholar 

  7. Bentin, S., McCarthy, G., & Wood, C. C. (1985). Event-related potentials, lexical decision, and semantic priming. Electroencephalography & Clinical Neurophysiology,60, 353–355.

    Google Scholar 

  8. Braeutigam, S., Swithenby, S. J., & Bailey, A. J. (2008). Contextual integration the unusual way: A magnetoencephalographic study of responses to semantic violation in individuals with autism spectrum disorders. European Journal of Neuroscience,27, 1026–1036.

    PubMed  Google Scholar 

  9. Brandwein, A. B., Foxe, J. J., Russo, N. N., Altschuler, T. S., Gomes, H., & Molholm, S. (2011). The development of audiovisual multisensory integration across childhood and early adolescence: A high-density electrical mapping study. Cerebral Cortex (New York),21(5), 1042–1055.

    Google Scholar 

  10. Brouwer, H., Fitz, H., & Hoeks, J. (2012). Getting real about semantic illusions: Rethinking the functional role of the P600 in language comprehension. Brain Research,1446, 127–143.

    PubMed  Google Scholar 

  11. Camblin, C. C., Ledoux, K., Boudewy, M., Gordon, P. C., & Swaab, T. Y. (2007). Processing new and repeated names: Effects of coreference on repetition priming with speech and fast RSVP. Brain Research,1146, 172–184.

    PubMed  Google Scholar 

  12. Coderre, E. (2019). Dismantling the “Visual Ease Assumption”: A review of visual narrative processing in clinical populations. Topics in Cognitive Science.

  13. Coderre, E. L., Chernenok, M., Gordon, B., & Ledoux, K. (2017). Linguistic and non-linguistic semantic processing in individuals with autism spectrum disorders: An ERP study. Journal of autism and developmental disorders,47(3), 795–812.

    PubMed  Google Scholar 

  14. Coderre, E. L., Cohn, N., Slipher, S. K., Chernenok, M., Ledoux, K., & Gordon, B. (2018). Brain and language visual and linguistic narrative comprehension in autism spectrum disorders: Neural evidence for modality-independent impairments. Brain and Language,186, 44–59.

    PubMed  Google Scholar 

  15. Cohn, N. (2013). Visual narrative structure. Cognitive Science,37(3), 413–452.

    PubMed  Google Scholar 

  16. Cohn, N. (2014). You’re a good structure, Charlie Brown: The distribution of narrative categories in comic strips. Cognitive Science,38(7), 1317–1359.

    PubMed  PubMed Central  Google Scholar 

  17. Cohn, N. (2019). Your brain on comics: A cognitive model of visual narrative comprehension. Topics in Cognitive Science, 1–35.

  18. Cohn, N., & Kutas, M. (2015). Getting a cue before getting a clue: Event-related potentials to inference in visual narrative comprehension. Neuropsychologia,77, 267–278.

    PubMed  PubMed Central  Google Scholar 

  19. Cohn, N., & Maher, S. (2015). The notion of the motion: The neurocognition of motion lines in visual narratives. Brain Research,1601, 73–84.

    PubMed  PubMed Central  Google Scholar 

  20. Cohn, N., & Kutas, M. (2017). What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive research: principles and implications,2 (1), 27.

    Google Scholar 

  21. Cohn, N. & Paczynski M. (2019). The neurophysiology of event processing in language and visual events. In R. Truswell (Ed.), Handbook of event structure. (pp. 624–637). Oxford: Oxford University Press.

    Google Scholar 

  22. Cohn, N., Paczynski, M., Jackendoff, R., Holcomb, P. J., & Kuperberg, G. R. (2012). (Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension. Cognitive psychology,65(1), 1–38.

    PubMed  PubMed Central  Google Scholar 

  23. Cohn, N., & Wittenberg, E. (2015). Action starring narratives and events: Structure and inference in visual narrative comprehension. Journal of Cognitive Psychology,27(7), 812–828.

    PubMed  Google Scholar 

  24. Coulson, S., King, J. W., & Kutas, M. (1998). Expect the unexpected: Event-related brain response to morphosyntactic violations. Language and Cognitive Processes,13, 21–58.

    Google Scholar 

  25. Dawson, M., Soulières, I., Ann Gernsbacher, M., & Mottron, L. (2007). The level and nature of autistic intelligence. Psychological Science,18, 657–662.

    PubMed  PubMed Central  Google Scholar 

  26. de Mello, C. B., Argollo, N., Shayer, B. P. M., Abreu, N., Godinho, K., Durán, P., et al. (2011). Versão abreviada do WISC-III: correlação entre QI estimado e QI total em crianças brasileiras. Psicologia: Teoria e Pesquisa, 27(2), 149–155

  27. Dunn, M. A., & Bates, J. C. (2005). Developmental change in neural processing of words by children with autism. Journal of Autism and Developmental Disorders,35, 361–376.

    PubMed  Google Scholar 

  28. Dunn, M., & Vaughn, H., Jr. (1999). Electrophysiologic correlates of semantic classification in autistic and normal children. Development Neuropsychology,16, 79–99.

    Google Scholar 

  29. Fishman, I., Yam, A., Bellugi, U., Lincoln, A., & Mills, D. (2011). Contrasting patterns of language associated brain activity in autism and Williams syndrome. Social Cognitive and Affective Neuroscience,6(5), 630–638.

    PubMed  Google Scholar 

  30. Friedrich, M., & Friederici, A. D. (2004). N400-like semantic incongruity effect in 19-month-olds: Processing known words in picture contexts. Journal of Cognitive Neuroscience,16(8), 1465–1477.

    PubMed  Google Scholar 

  31. Ganis, G., Kutas, M., & Sereno, M. I. (1996). The search for “common sense”: An electrophysiological study of the comprehension of words and pictures in reading. Journal of Cognitive Neuroscience,8(2), 89–106.

    PubMed  Google Scholar 

  32. Goldstein, G., Beers, S. R., Siegel, D. J., & Minshew, N. J. (2001). A comparison of WAIS—R profiles in adults with high-functioning autism or differing subtypes of learning disability. Applied Neuropsychology,8(3), 148–154.

    PubMed  Google Scholar 

  33. Gunter, T. C., Stowe, L. A., & Mulder, G. (1997). When syntax meets semantics. Psychophysiology,34, 660–676.

    PubMed  Google Scholar 

  34. Hagoort, P., Brown, C., & Groothusen, J. (1993). The syntactic positive shift (SPS) as an ERP measure of syntactic processing. Lang. Cogn. Process,8, 439–483.

    Google Scholar 

  35. Hahne, A., Eckstein, K., & Friederici, A. D. (2004). Brain signatures of syntactic and semantic processes during development. Journal of Cognitive Neuroscience,16(7), 1302–1318.

    PubMed  Google Scholar 

  36. Hahne, A., & Friederici, A. D. (1999). Electrophysiological evidence for two steps in syntactic analysis: Early automatic and late controlled processes. Journal of Cognitive Neuroscience,11, 194–205.

    PubMed  Google Scholar 

  37. Holland, S. K., Vannest, J., Mecoli, M., Jacola, L. M., Tillema, J.-M., Karunanayaka, P. R., et al. (2007). Functional MRI of language lateralization during development in children. International Journal of Audiology,46, 533–551.

    PubMed  PubMed Central  Google Scholar 

  38. Johnels, J. Å., Hagberg, B., Gillberg, C., & Miniscalco, C. (2013). Narrative retelling in children with neurodevelopmental disorders: Is there a role for nonverbal temporal-sequencing skills? Scandinavian Journal of Psychology,54(5), 376–385.

    PubMed  Google Scholar 

  39. Kamio, Y., & Toichi, M. (2000). Dual access to semantics in autism: Is pictorial access superior to verbal access? Journal of Child Psychology and Psychiatry,41, 859–867.

    PubMed  Google Scholar 

  40. Kaufman, A. S., & Lichtenberger, E. O. (2006). Assessing adolescent and adult intelligence (3rd ed.). Hoboken: Wiley.

    Google Scholar 

  41. Kujala, T., Lepistö, T., & Näätänen, R. (2013). The neural basis of aberrant speech in autism spectrum disorders. Neuroscience & Biobehavioral Reviews,37, 1–8.

    Google Scholar 

  42. Kuperberg, G., Sitnikova, T., Caplan, D., and Holcomb, P. (2003). Electrophysiological distinctions in processing conceptual relation- ships within simple sentences. Cogn. Brain Res,17, 117–129.

    Google Scholar 

  43. Kuperberg, G. (2007). Neural mechanisms of language comprehension: Challenges to syntax. Brain Research,1146, 23–49.

    PubMed  Google Scholar 

  44. Kuperberg, G. R. (2013). The pro-active comprehender: What event-related potentials tell us about the dynamics of reading comprehension. In B. Miller, L. Cutting, & P. McCardle (Eds.), Unraveling the behavioral, neurobiological, and genetic components of reading comprehension (pp. 176–192). Baltimore: Paul Brookes Publishing.

    Google Scholar 

  45. Kuperberg, G. R. (2016). Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events. Language, Cognition and Neuroscience,31(5), 602–616.

    PubMed  PubMed Central  Google Scholar 

  46. Kutas, M., & Federmeier, K. D. (2011). Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP). Annual Review of Psychology,62, 621–647.

    PubMed  PubMed Central  Google Scholar 

  47. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science,207, 203–208.

    PubMed  Google Scholar 

  48. Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature,307, 161–163.

    PubMed  Google Scholar 

  49. Lee, C. L., & Federmeier, K. D. (2008). To watch, to see, and to differ: An event-related potential study of concreteness effects as a function of word class and lexical ambiguity. Brain and Language,104, 145–158.

    PubMed  Google Scholar 

  50. Lepistö, T., Kujala, A., Vanhala, R., Alku, P., Huotilainen, M., & Näätänen, R. (2005). The discrimination of and orienting to speech and non-speech sounds in children with autism. Brain Research,1066, 147–157.

    PubMed  Google Scholar 

  51. Lord, C., Rutter, M., & Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders,24, 659–685.

    PubMed  Google Scholar 

  52. Loschky, L., Hutson, J., Smith, M., Magliano, J., & Smith, T. (2017). Viewing Static Visual Narratives Through the Lens of the Scene Perception and Event Comprehension Theory (SPECT) "Viewing Static Visual Narratives Through the Lens of the Scene Perception and Event Comprehension Theory (SPECT)". In A. Dunst, J. Laubrock, & J. Wildfeuer (Eds.), Empirical Comics Research: Digital, Multimodal, and Cognitive Methods (pp. 217–238). London: Routledge.

    Google Scholar 

  53. Manfredi, M., Adorni, R., & Proverbio, A. M. (2014). Why do we laugh at misfortunes? An electrophysiological exploration of comic situation processing. Neuropychologia,61, 324–334.

    Google Scholar 

  54. Manfredi, M., Cohn, N., De Araújo Andreoli, M., & Boggio, P. S. (2018). Listening beyond seeing: Event-related potentials to audiovisual processing in visual narrative. Brain and Language,185, 1–8.

    PubMed  Google Scholar 

  55. Manfredi, M., Cohn, N., & Kutas, M. (2017). When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative. Brain and Language,169, 28–38.

    PubMed  PubMed Central  Google Scholar 

  56. Männel, C., & Friederici, A. D. (2011). Intonational phrase structure processing at different stages of syntax acquisition: ERP studies in 2-, 3-, and 6-year-old children. Developmental Science,14(4), 786–798.

    PubMed  Google Scholar 

  57. McCleery, J. P., Ceponiene, R., Burner, K. M., Townsend, J., Kinnear, M., & Schreibman, L. (2010). Neural correlates of verbal and nonverbal semantic integration in children with autism spectrum disorders. Journal of Child Psychology and Psychiatry and Allied Disciplines,51(3), 277–286.

    Google Scholar 

  58. McPherson, W. B., & Holcomb, P. J. (1999). An electrophysiological investigation of semantic priming with pictures of real objects. Psychophysiology,36(1), 53–65.

    PubMed  Google Scholar 

  59. Nakazawa, J. (2005). Development of manga (comic book) literacy in children. In D. W. Shwalb, J. Nakazawa, & B. J. Shwalb (Eds.), Applied developmental psychology: Theory, practice, and research from Japan (pp. 23–42). Greenwich, CT: Information Age Publishing.

    Google Scholar 

  60. Nakazawa, J. (2016). Manga literacy and manga comprehension in Japanese children. In N. Cohn (Ed.), The visual narrative reader (pp. 157–184). London: Bloomsbury.

    Google Scholar 

  61. O’Connor, K. (2012). Auditory processing in autism spectrum disorder: A review. Neuroscience and Biobehavioral Reviews,36(2), 836–854.

    PubMed  Google Scholar 

  62. Olivares, E. I., Iglesias, J., & Bobes, M. A. (1999). Searching for face-specific long latency ERPs: A topographic study of effects associated with mismatching features. Cognitive Brain Research,7, 343–356.

    PubMed  Google Scholar 

  63. Osterhout, L., & Holcomb, P. (1992). Event-related brain potentials elicited by syntactic anomaly. Journal of Memory and Language,31, 785–806.

    Google Scholar 

  64. Pijnacker, J., Geurts, B., van Lambalgen, M., Buitelaar, J., & Hagoort, P. (2010). Exceptions and anomalies: An ERP study on context sensitivity in autism. Neuropsychologia,48(10), 2940–2951.

    PubMed  Google Scholar 

  65. Proverbio, A. M., & Riva, F. (2009). RP and N400 ERP components reflect semantic violations in visual processing of human actions. Neuroscience Letters,459, 142–146.

    PubMed  Google Scholar 

  66. Quante, L., Bölte, J., & Zwitserlood, P. (2018). Dissociating predictability, plausibility and possibility of sentence continuations in reading: Evidence from late-positivity ERPs. PeerJ 1–24.

  67. Ribeiro, T. C., Valasek, C. A., Minati, L., & Boggio, P. S. (2013). Altered semantic integration in autism beyond language: A cross-modal event-related potentials study. NeuroReport,24(8), 414–418.

    PubMed  Google Scholar 

  68. Sahyoun, C. P., Soulières, I., Belliveau, J. W., Mottron, L., & Mody, M. (2009). Cognitive differences in pictorial reasoning between high-functioning autism and Asperger’s syndrome. Journal of Autism and Developmental Disorders,39(7), 1014–1023.

    PubMed  PubMed Central  Google Scholar 

  69. Schneider, J. M., Abel, A. D., Ogiela, D. A., Middleton, A. E., & Maguire, M. J. (2016). Developmental differences in beta and theta power during sentence processing. Developmental Cognitive Neuroscience,19, 19–30.

    PubMed  PubMed Central  Google Scholar 

  70. Sitnikova, T., Holcomb, P. J., Kiyonaga, K. A., & Kuperberg, G. R. (2008). Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. Journal of Cognitive Neuroscience,20, 1–21.

    Google Scholar 

  71. Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003). Semantic integration in videos of real-world events: An electrophysiological investigation. Psychophysiology,40(1), 160–164.

    PubMed  Google Scholar 

  72. Soulières, I., Dawson, M., Samson, F., Barbeau, E. B., Sahyoun, C. P., Strangman, G. E., et al. (2009). Enhanced visual processing contributes to matrix reasoning in autism. Human Brain Mapping,30, 4082–4107.

    PubMed  PubMed Central  Google Scholar 

  73. Tanner, D., & Van Hell, J. G. (2014). ERPs reveal individual differences in morphosyntactic processing. Neuropsychologia,56, 289–301.

    PubMed  Google Scholar 

  74. Van Berkum, J. J., Hagoort, P., & Brown, C. M. (1999). Semantic integration in sen- tences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience,11(6), 657–671.

    PubMed  Google Scholar 

  75. Van Berkum, J. J., Zwitserlood, P., Hagoort, P., & Brown, C. M. (2003). When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. Cognitive Brain Research,17(3), 701–718.

    PubMed  Google Scholar 

  76. Van Petten, C., & Kutas, M. (1991). Influences of semantic and syntactic context on open and closed class words. Memory and Cognition,19, 95–112.

    PubMed  Google Scholar 

  77. Van Petten, C., & Luka, B. J. (2006). Neural localization of semantic context effects in electromagnetic and hemodynamic studies. Brain and Language,97, 279–293.

    PubMed  Google Scholar 

  78. Van Petten, C., & Luka, B. J. (2012). Prediction during language comprehension: Benefits, costs, and ERP components. International Journal of Psychophysiology,83, 176–190.

    PubMed  Google Scholar 

  79. Wechsler, D. (1981a). WAIS-R manual: Wechsler adult intelligence scale-revised. Cleveland, OH: Psychological Corporation.

    Google Scholar 

  80. Wechsler, D. (1981b). Wechsler Adult Intelligence Scale—Revised. Cleveland, OH: Psychological Corporation.

    Google Scholar 

  81. Wechsler, D. (1992). Wechsler Intelligence Scale for Children (3rd edn rev). NewYork: Psychological Corporation.

    Google Scholar 

  82. Wechsler, D. (1999). Wechsler Intelligence Scale for Children. New York: Psychological Corporation.

    Google Scholar 

  83. West, W. C., & Holcomb, P. J. (2002). Event-related potentials during discourse-level semantic integration of complex pictures. Cognitive Brain Research,13, 363–375.

    PubMed  Google Scholar 

  84. Zalla, T., Labruyère, N., Clément, A., & Georgieff, N. (2010). Predicting ensuing actions in children and adolescents with autism spectrum disorders. Experimental Brain Research,201, 809–819.

    PubMed  Google Scholar 

Download references

Acknowledgments

Mirella Manfredi was supported by a FAPESP post-doctoral research grant (2015/00553-5). Paulo S. Boggio is supported by a CNPq research grant (311641/2015-6).

Author information

Affiliations

Authors

Contributions

M.M. conceived and designed the study, analyzed data and wrote the paper. M.M, N.C. and P.S.B. interpreted the data. E.F. and P. S. M.developed stimuli, gathered behavioral pilot data and EEG data.

Corresponding author

Correspondence to Mirella Manfredi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Manfredi, M., Cohn, N., Sanchez Mello, P. et al. Visual and Verbal Narrative Comprehension in Children and Adolescents with Autism Spectrum Disorders: An ERP Study. J Autism Dev Disord 50, 2658–2672 (2020). https://doi.org/10.1007/s10803-020-04374-x

Download citation

Keywords

  • Autism
  • Semantic processing
  • N400
  • LP
  • Visual narrative