Abstract
Language depends critically on the integration of lexical information across multiple words to derive semantic concepts. Limitations of spatiotemporal resolution have previously rendered it difficult to isolate processes involved in semantic integration. We utilized intracranial recordings in epilepsy patients (n = 58) who read written word definitions. Descriptions were either referential or non-referential to a common object. Semantically referential sentences enabled high frequency broadband gamma activation (70–150 Hz) of the inferior frontal sulcus (IFS), medial parietal cortex, orbitofrontal cortex (OFC) and medial temporal lobe in the left, language-dominant hemisphere. IFS, OFC and posterior middle temporal gyrus activity was modulated by the semantic coherence of non-referential sentences, exposing semantic effects that were independent of task-based referential status. Components of this network, alongside posterior superior temporal sulcus, were engaged for referential sentences that did not clearly reduce the lexical search space by the final word. These results indicate the existence of complementary cortical mosaics for semantic integration in posterior temporal and inferior frontal cortex.
Similar content being viewed by others
Introduction
The neurobiology of human sentence processing brings with it major implications for our understanding of the organization and timing of cortical computation. It still remains unclear whether multiple cortical regions are needed to evaluate sentence meaning1,2, or if highly overlapping heterogeneous constituents within a single region can encode distinct aspects of meaning3. Retrieving specific words from memory to refer to objects in the world is an essential contribution made by language to our cognitive abilities4,5,6,7,8,9,10,11,12. The fundamentally generative nature of language enables us to derive concepts using cues we may never have encountered, such as when the phrase “an elongated yellow fruit” is immediately recognized as referring to a banana. Such derivations depend upon the rapid integration of multiple lexical objects into a larger structure, but this crucially occurs alongside other semantic processes13,14. We know broadly that language-related semantic processes engage the posterior temporal lobe15,16,17 and prefrontal and parietal cortices18,19,20, but there remains no general consensus in the literature. Much work has focused on picture naming (overt reference) rather than naming to a definition (inferential meaning)21. Given the very rapid and distributed nature of these processes22,23, previous research into lexical access has been unable to disentangle retrieval from related computations, such as inferring semantic coherence. Much work into linguistic coherence has utilized scalp event-related potentials or functional MRI (fMRI)24,25, which lack the fine spatiotemporal resolution needed to comprehensively map cortical responses. Under some analyses, the cortical substrates for language and semantic processing overlap26,27, while others point to dissociability28. Across inferior frontal, inferior parietal, and posterior temporal cortices, it remains unclear if there are functionally specialized subregions for diverse semantic processes29,30,31.
In order to isolate sites involved in distinct semantic processes we used an orthographic sentence comprehension and linguistic reference paradigm in a large cohort of subjects undergoing intracranial electrocorticography for the evaluation of medically refractory epilepsy (iEEG) (58 patients; 11,328 electrodes)32,33. We used broadband high gamma activity (BGA; 70–150 Hz), which strongly correlates with the fMRI blood oxygen level-dependent (BOLD) signal, to index local cortical processing34. We comprehensively mapped BGA responses across the language-dominant cortex by presenting sentences triggering inferential semantics (as opposed to directly showing a picture of an object) that were either referential35 or non-referential to common objects36.
Patients generated common object names in response to written descriptions of variable lengths via rapid serial visual presentation (500 ms per word). Our orthographic stimuli afforded temporally precise lexical inputs and minimize integrative processes that are intrinsic to continuous auditory or visual inputs. The final word dictated whether the sentence referred to a common lexical item, manipulating the referential nature of the description. Within non-referential trials, we manipulated their semantic coherence. This allowed us to isolate cortical dynamics of semantic convergence to a common lexical item. To illustrate these concepts, the sentences “The part of the tree that grows underground” and “A person at the circus who makes you laugh” permit successful lexical access (e.g., referring to roots and clown). However, “A person at the circus who makes you commute” does not permit lexical access to a common name. Yet, this sentence retains semantic coherence and contains what we will call a weak violation of lexico-semantic rules (i.e., it is conceptually possible that there are people at circuses who make other people commute, but there is no common word for this concept). In contrast, “A place where oceans shop” is non-referential to a common word but is also semantically incoherent, and represents a strong violation (see Methods). For all Non-Referential trials, patients were instructed to verbally respond “nonsense”. As such, our notion of referential meaning conforms to a broader sense commonly used in the psycholinguistics literature pertaining to inferential semantics and lexical reference, as opposed to a narrower sense pertaining to definiteness and grammatical specificity regulated by functional syntactic structure.
Lastly, we collected norming data (n = 80) to quantify the point of what we term semantic ‘narrowing’ that enabled an analysis of the timing and extent of the lexicon search effort for all Referential sentences, e.g., “It’s white and falls from the sky in winter” was likely to be inferred as snow before the final word, while other trials did not enable inference until the final word, e.g., “An object used for weighing”. This pertains to the notion of cloze probability or the extent to which a particular item is predicted from a context, and likely indexes composition-driven lexical search effort and hence allows us to explore an additional feature of lexical access37,38,39.
To summarize our conditions, on the one hand, we have integrative lexical access (‘Reference’) and a measure of how effortful this is (‘Narrowing’), and on the other hand, we have semantic coherence (splitting up Non-Referential trials into coherent vs. incoherent). Our design hence afforded four main experimental conditions, focusing primarily on the effects of semantic reference and coherence (Fig. 1d) (Non-Referential: coherent vs. incoherent; Referential: strong narrowing vs. limited narrowing). These two semantic processes are likely to be somewhat correlated but expected to be engaged to varying degrees during sentence comprehension, depending on the particular task and linguistic content. It is an open question whether these processes engage overlapping or non-overlapping cortex at parallel or distinct time intervals. Together, these analyses enabled us to comprehensively elaborate the spatiotemporal dynamics of semantic integration.
Results
Behavioral performance
Individual reaction time (Fig. 1) averaged 1765 ms (SD: 680 ms) after the offset of the final word in the sentence. Referential trials had significantly faster articulation reaction times than non-referential trials (paired t-test; t(1,83): 1.86, p-value: 0.032).
Spatiotemporal dynamics of orthographic sentence processing
To probe the build-up of local cortical activity across successive words, we generated a population-level map of cortical activity using a surface-based mixed-effects multi-level analysis (SB-MEMA)34,40,41,42. This revealed serially increasing activation across the sentence duration in a distributed orthographic sentence processing network (Fig. 2, top). Early activation was found in the inferior frontal gyrus (IFG), medial parietal cortex (MPC), anterior temporal lobe (ATL), and posterior middle temporal gyrus (pMTG), and this was followed by late activation in the ventromedial prefrontal cortex (vmPFC), posterior cingulate and orbitofrontal cortex (OFC). Three regions – the ventral temporal cortex, the inferior lateral temporo-occipital cortex, and the inferior frontal sulcus (IFS) in its entire antero-posterior extent – were active throughout sentence reading and showed a clear increase in BGA at the final word and a broader spread of activity across the sentence (Fig. 2, bottom). Contrasts with the final word against both the first word and the penultimate word further highlight regions that exhibited a relative increase over the course of the trial (Fig. 2, bottom). Non-Referential trials were excluded from analyses involving the final word.
Reference to common objects
A comparison of referential vs. non-referential trials at the onset of the final word revealed greater BGA for referential trials in MFG and middle IFS (500–700 ms: β = 0.08 (SD: 0.03); p < 0.001; 700–900 ms: β = 0.10 (SD: 0.05); p < 0.001), MPC and parahippocampal cortex (500–700 ms: β = 0.11 (SD: 0.06); p < 0.001; 700–900 ms: β = 0.13 (SD: 0.08); p < 0.001), vmPFC (500–700 ms: β = 0.07 (SD: 0.01); p < 0.001; 700–900 ms: β = 0.08 (SD: 0.04); p < 0.001), and OFC (500–700 ms: β = 0.09 (SD: 0.04); p < 0.001; 700–900 ms: β = 0.11 (SD: 0.05); p < 0.001) (Fig. 3, Supplementary Figs. 1, 2, 4). Non-referential trials exhibited BGA increases relative to referential trials in posterior superior temporal cortex (500–700 ms: β = 0.07 (SD: 0.01); p < 0.001) and anterior inferior frontal gyrus (aIFG) (500–700 ms: β = 0.08 (SD: 0.03); p < 0.001) (Fig. 3). These effects were specific to the period immediately after final word onset, as opposed to patient verbal articulation (Fig. 3b). Frontal (Fig. 3c) and medial temporal (Fig. 3e) effects of reference, and an exemplar posterior temporal electrode displaying this non-referential sensitivity (Fig. 3d), are plotted. Across the non-dominant right hemisphere, we also saw general increases for non-referential trials relative to referential trials (Supplementary Fig. 3), although given our significantly reduced electrode coverage over the right hemisphere we refrain from any further observations.
Semantic coherence
Focusing on non-referential sentences, a whole brain analysis contrasting coherent and incoherent trials showed increased activity for incoherent non-referential sentences in the medial frontal cortex (300–500 ms after the onset of the final word: β = 0.10 (SD: 0.04); p = 0.001) and superior medial parietal cortex (300–500 ms: β = 0.09 (SD: 0.03); p < 0.001). The supero-medial parietal activations were seen in loci distinct from with the sites sensitive to referential vs. non-referential contrasts (Fig. 3f). Coherent non-referential sentences resulted in greater BGA in IFS (300–500 ms: β = 0.08 (SD: 0.03); p = 0.001), aIFG (700–900 ms: β = 0.11 (SD: 0.05); p = 0.001) (Fig. 3G), angular gyrus (700–900 ms: β = 0.06 (SD: 0.01); p < 0.001), pMTG (700–900 ms: β = 0.08 (SD: 0.03); p = 0.002) and OFC (700–900 ms: β = 0.07 (SD: 0.01); p = 0.001).
Integrative lexical access
To calibrate our stimuli, we conducted a norming study with native English speakers (n = 80) enabling the derivation of the point of what we call semantic narrowing, elaborating further on the above process of reference resolution and lexical access. This denotes the probability that the defined object could be identified before the presentation of the final word. Certain sentences yielded probable answers before the final word due to the presence of semantically salient information (e.g., “Something you use to unlock a door”), whereas other sentences were ambiguous until the final word (e.g., “What you use to measure temperature”). We then contrasted trials that exhibited limited semantic narrowing (i.e., they provided a smaller set of possible answers to the definition before the final word) at the point of final word onset with those exhibiting strong narrowing (Supplementary Fig. 5). This was the most conservative means of isolating a contrast between limited and strong narrowing due to variability in the position of narrowing words throughout the sentence, as well as our inability to ensure that patients were specifically constraining their definitional search at time points concurrent with the mid-sentence positions in the norming data. Articulation reaction times (Supplementary Fig. 5A) did not differ between narrowing conditions (strong narrowing: 1886 ± 708 ms; limited narrowing: 1904 ± 699 ms; paired t-test, t(1,34) = −0.09, p = 0.46), which possibly suggests distinct mechanisms from those implicated in cloze tasks38. There was no significant difference between conditions with respect to sentence length (strong narrowing length; M: 6.8; range: 5 (4–9); limited narrowing length; M: 6.4; range: 5 (3–8); paired t-test, t(1,41) = 1.36, p = 0.08). There was also no significant difference in the frequency of the final word between the two conditions (strong narrowing final word frequency, SUBTLEXus log-frequency (Lg10CD) M: 3.09; range: 2.81 (1.11–3.92); limited narrowing final word M: 2.90; range: 2.7 (1.17–3.91); paired t-test, t-(1,41) = −1.2; p = 0.240), and the cortical loci we document below are not typically implicated in lexical frequency sensitivity43.
Referential trials with limited semantic narrowing revealed greater BGA than strong narrowing trials in and around posterior superior temporal sulcus (pSTS) beginning approximately 250 ms after the onset of the final word and lasting until approximately 900 ms (300–500 ms: β = 0.10 (SD: 0.04); p = 0.001; 500–700 ms: β = 0.08 (SD: 0.03); p < 0.001) (Supplementary Fig. 5b). Greater BGA for limited narrowing trials was also found in MPC (500–700 ms: β = 0.12 (SD: 0.07); p = 0.003), IFS (300–500 ms: β = 0.11 (SD: 0.05); p = 0.008; 500–700 ms: β = 0.12 (SD: 0.07); p = 0.004), anterior temporal lobe (500–700 ms: β = 0.06 (SD: 0.01); p = 0.004) and OFC (500–700 ms: β = 0.12 (SD: 0.07); p < 0.001). Spectrograms depicting effects in IFS and pSTS are plotted (Supplementary Fig. 5C), alongside exemplar electrodes from pSTS (Supplementary Fig. 5D).
Discussion
Our evaluation of the spatiotemporal dynamics of sentence reading identified distinct semantic roles for closely adjacent loci in the lateral inferior frontal and posterior temporal cortex. We examined the effects of integrative lexical access and semantic coherence. Whilst frontal gyral structures are commonly implicated in the literature on linguistic semantics2,15, we discovered that the inferior frontal sulcus (IFS) exhibited greater high gamma activity for all semantic contrasts, exhibiting a clear mosaic-like patchwork of activity across its sub-regions (Fig. 4). Although many models have proposed anterior-posterior distinctions for inferior frontal regions with respect to certain higher-order language functions15,44,45,46, our recordings of distinct semantic effects provide a more complex picture. Though we indeed find support for separable semantic effects in anterior and posterior inferior frontal gyrus (IFG) – coherence for the former, and difficulty of integrative lexical access for the latter – there was also found to be some functional overlap (Fig. 4). Neighboring sulcal loci also showed effects that were not only earlier than those in IFG, but also more topographically complex. Centered around IFS and spreading out dorsally towards the middle frontal gyrus and ventrally towards the anterior/posterior inferior frontal gyrus, we uncovered a cortical mosaic of functional sensitivity to reference and coherence, in addition to finding effects of reference resolution difficulty (‘narrowing’). Early processing windows implicated IFS in all aspects of semantics, and in later windows sub-portions of IFS exposed a clearer functional tessellation with some remaining overlap. The orbitofrontal cortex (OFC) was also sensitive to all semantic processes, albeit during a later time window. OFC also became significantly active later in sentence reading, whereas IFS was active at all periods (Fig. 2).
We additionally discovered that the posterior superior temporal sulcus (pSTS) and posterior middle temporal gyrus (pMTG) jointly contributed to all semantic processes. The temporal progression of sensitivity in the posterior temporal cortex had a clear delineation between early effects of reference (in particular, difficult reference via ‘limited narrowing’), and finally coherence. Portions of frontotemporal cortex were engaged for all aspects of linguistic semantics but became rapidly dedicated to certain processes at distinct times. Previous intracranial work has implicated pSTS in the initial construction of meaningful phrases, and IFS in the later evaluation of these phrases3. Our results further highlight the joint role of pSTS-IFS in semantic composition demands, with early semantic narrowing effects in pSTS (onset ~250 ms) being followed by effects for all conditions in IFS (~500 ms) (Fig. 4). The temporal progression of early high gamma increases for non-referential over referential items in pSTS, to frontal effects of semantic coherence, is concordant with MEG studies showing the spread of sentence-closure N400 congruity signatures from STS to frontal cortex47. Alongside these sites, we found that medial parietal cortex (MPC), hippocampus, parahippocampal gyrus (PHG), OFC, intraparietal sulcus, and ventromedial prefrontal cortex (vmPFC) are engaged during the processing of sentences that permitted inference to common object names.
The network typically implicated in the ability to successfully generate names of visually presented common objects consists of the inferior frontal cortex, mid-fusiform cortex, and the supplementary motor area48,49,50,51. Regions discovered from fMRI that are involved in lexical access for both picture-derived and sentence-derived access are mid-fusiform cortex, inferior frontal gyrus, and middle temporal gyrus52. Our results provide additional details about particular integrative components of semantic processing, with our analyses dissociating lexical search difficulty (within referential trials) and semantic coherence (within non-referential trials). We also reported on the basic build-up of local activity across normal sentence processing (Fig. 2), exposing regions more likely to be involved in lexical access from the Final Word – Word 1 contrast, and regions more likely to be involved in semantic integration from the Final Word – Penultimate Word contrast. We assume that the major nodes we identified in this network are responsible for distinct components of semantic integration, with areas such as pMTG, posterior cingulate, OFC, and vmPFC becoming significantly active only later in the sentence. As such, processes relating to situation model construction may be subserved by these regions, with vmPFC being implicated recently in associative inference and memory integration53. We further discuss below the likely contributions of regions implicated in our results to orthographic sentence processing in the service of lexical access (Fig. 4).
Previous research into linguistic reference to both visual objects and sounds implicated greater engagement of MPC from 500–600 ms after the orthographic presentation of the referential word in a sentence54. These regions form part of the default network, which is engaged during endogenous attentional tasks (i.e., directing attention at ‘internal’ processes and events, in contrast to ‘external’ data) and interoceptive monitoring, with representational search likely being greater for sentences permitting successful selection of objects from the lexicon55. Retrosplenial cortex forms a major part of this MPC response. This region encodes (i) predictions and error corrections for current sensory states with internal representations of the environment, and (ii) shifting perspectives and mental reference frames, both of which support endogenous attention and likely facilitate task-related representational search56. The clear involvement of MPC across a range of semantic integration processes may relate to the recently documented human-specific sulcal morphology of posteromedial cortex57. Default network activity is related to processing rich representations of events, either real or imagined, of the kind inferred by patients in our task. Relatedly, our effects in dorsal frontal sites seem to overlap with a node in the multiple demand network58,59, which has been implicated in linguistic processing difficulty, seemingly congruent with our semantic narrowing effects in IFS and MFG.
Linguistic reference to coherent entities has been shown to implicate medial frontal and medial parietal regions, along with perisylvian language regions19. Sentence pairs introducing a conjoined subject (e.g., “John and Mary”) result in greater MPC engagement compared to individual referents60. Referentially ambiguous pronouns (e.g., “Saul told Mike that he…”) have been shown to modulate medial parietal activity18. Our findings concerning medial parietal regions seem in accord with the involvement of the MPC in the posterior medial system61, which is part of the distributed memory network and is involved in context memory, constructing situation models, and combining concepts from distinct categorical domains62,63,64,65,66. The MPC has been shown to contain selective regions for the recall of information about familiar people and places but does not demonstrate activity during visual object naming67,68, and the precuneus has been implicated in the encoding of complex memories and actions69. Our results indicate involvement not just in the above forms of grammatical co-reference, but also in inferential semantics pertaining to basic lexical access.
In addition to MPC, the posterior medial system incorporates the hippocampus and PHG. Hippocampal theta tracks the amount of contextual linguistic information in a sentence, such that in predictable sentence contexts theta power increases during sentence presentation70. Other recent intracranial work has indicated a role for the medial temporal lobe in verbal working memory71. This suggests that portions of the hippocampus and PHG actively relate incoming words to sentence contexts and contribute memory-related representations needed to successfully search the (narrowed) lexicon. We discovered early hippocampal theta power increases for referential trials (100–400 ms) followed directly by BGA increases in PHG (300–1000 ms); it seems plausible that joint MPC-hippocampal engagement underlies contextual associations72,73,74.
Our results are also concordant with the findings that hippocampal-complex damage leads to impairments in semantic association tasks testing words learned prior to damage75, and that hippocampal theta power increases during episodic memory retrieval76. It has been argued that the hippocampus might contribute to language’s facility for displacement77, or the capacity to refer to objects/events outside the current spatiotemporal context, and also the tracking of situation/context boundaries78, potentially concordant with its present involvement in inferential semantics.
Some of our stimuli that did not refer to a common object were semantically coherent, while others were incoherent (Fig. 1). When contrasting these non-referential trials, we discovered that coherent sentences resulted in greater BGA in aIFG, IFS, angular gyrus, OFC and pMTG relative to incoherent sentences. This effect in aIFG (onset around 300 ms) possibly indexes the entry of the final word into the workspace and the successful wrap-up effect for semantically legal structures, and the subsequent control of the appropriate lexical category15,79. Previous work has implicated IFG in various semantic processes during different tasks such as plausibility and acceptability judgements44,80. Our results are generally in line with the role of IFG in effortful lexico-semantic processing81, but afford a more nuanced picture: Lateral aIFG exhibits greater BGA for non-referential relative to referential sentences, and neighboring aIFG and IFS show greater BGA for semantically coherent non-referential sentences, relative to semantically incoherent non-referential sentences. This leads to a fine-grained spatiotemporal map of anterior IFG activity. Lateral aIFG activity marks referential violations and cortex abutting IFS responds to semantic coherence. We note that during sentence comprehension (Fig. 2) these portions in aIFG and IFS are engaged jointly, suggestive of coordinated semantic processes, dissociable with fine spatiotemporal resolution and appropriate behavioral contrasts. Indeed, there are also a number of semantic integration processes that fall outside of our current focus on reference and coherence14, and we expect future intracranial work can help isolate even further the spatiotemporal dynamics of the semantics network.
We also note that inferior parietal regions (angular gyrus) were implicated broadly in semantic coherence, with a smaller region being implicated in both semantic coherence and effortful reference. This region was also only moderately active towards the end of sentence comprehension, as in a previous MEG study of sentence processing (Fig. 1)82. Though not as engaged as other regions, these results support suggestions that this region codes for event/thematic structure15,83 and may be involved in more incremental aspects of event semantics rather than grammatical processing, thus being recruited for “coherent sentence meaning”, as Matchin and colleagues claim82.
Referential trials that exhibited a limited degree of semantic ‘narrowing’ (i.e., that did not clearly reduce the space of possible lexical items to select for articulation) resulted in greater recruitment of IFS, pSTS, MPC, and ATL. Recruitment of pSTS was earliest (approximately 250 ms) and may pertain to composition-related demands or lexical search effort, indexing greater engagement of compositional processing due to greater integrative lexical search demands15,84, in particular given previous intracranial work implicating this region in basic semantic composition and lexicality3, and suggestions that pSTS acts as an interface between externalization and syntactic/semantic representations85. ATL engagement is in line with its apparent role in conceptual processing and entity-related (e.g., common object) representations86. Though our two ‘narrowing’ conditions did not strictly isolate the more fundamental processing stage of lexical access (since ‘limited narrowing’ sentences afford varying degrees of lexical access facilitation), we note that future research with other paradigms aimed at definitively disambiguating lexical access from semantic processes is needed to explore this. Certainly, it seems clear from prior lesion studies that patients with chronic left prefrontal cortex damage exhibit impaired lexical selection76. We also note that the closely interwoven anatomical localization of the reference and narrowing effects indicates the existence of a ‘core’ integrative lexical access network, flanked by ‘peripheral’ regions that are recruited in cases of greater referential difficulty (i.e., ‘limited narrowing’ trials). For example, consider the organization of effects in posterior temporal, inferior frontal, and medial parietal areas, where regions sensitive to both reference and narrowing are flanked by regions only sensitive to reference and/or narrowing.
We did not attempt here to map out the word-by-word dynamics of easy/difficult reference resolution, and we instead isolated the time window where we could most confidently isolate differences in integrative lexical access. We recognize the limitations of this analysis, and the clear dynamics we depict in our analyses are conservatively interpreted here as pertaining to some aspect of the reference resolution process (i.e., coding for semantically salient information, or some process of set-reduction over the course of the sentence).
Recent intracranial work37 reported effects of sensitivity to cloze probability in Broca’s area in six patients during a sentence completion production task. Our results suggest that a smaller region of the frontal cortex (middle IFS) is engaged during lexico-semantic search, not necessarily tied to next-word statistics. In addition, in contrast to Wang and colleagues who reported on a small cohort of patients with subdural arrays, we used a combination of subdural grids and depth electrodes for extensive gyral and sulcal coverage in a large cohort. Other recent intracranial work points towards IFS sensitivity to lexical selectivity during single-word production87.
While medial brain structures often get overlooked in models of higher-order language, our results suggest that a more extended network is involved in elementary components of sentence comprehension and lexical access. We provide a general model for the neural representation of linguistic meaning derived from a conjunction map of the above analyses (Fig. 4), highlighting also general engagement in processes pertaining to higher-order semantics and relevant to the present task context. While the annotated descriptions are inferred based on contrasts of interest (see below), the cortical surface maps and their coloration are derived from a conjunction SB-MEMA analysis (see Methods, and figure legend). We plot effects pertaining to the two central psycholinguistic processes explored in this article: integrative lexical access and the derivation of semantic coherence. The effects of reference, and the related measure of narrowing, are plotted alongside the effects of coherence (Fig. 4).
Our results can also be viewed in the context of the three core components of linguistic meaning – Types, Tokens, and Referents. A type is a general category of an entity (e.g., flower), a token is a particular concrete instantiation (e.g., a specific flower), and a referent arises via the explicit denotation of a token (e.g., “That yellow thing in the garden”). Previous work has claimed that these components significantly implicate left temporal (Types), inferior frontal (Tokens), and inferior parietal (Referents) cortices1. However, our results motivate some potential revisions to this architecture. Our paradigm involved descriptions of referents calling upon conceptions of general types. However, our paradigm was not designed to specifically adjudicate between these three aspects of meaning, and so our association between semantic components and conditional effects should be taken as purely preliminary and speculative, intended to direct future inquiry and to map out areas of the brain that we can at least rule in with respect to semantic components. That is to say, we cannot definitively rule out the involvement of the posterior temporal cortex in, for example, establishing referents; we can only rule it in with respect to being recruited for integrative lexical access. Nevertheless, the immediate compatibility here appears to be between our parietal effects and the connections with processing the discourse referent. MPC is predominantly sensitive to reference and narrowing and may form part of the lexical search process and the construction of Referents. We note that although this region is involved in general category sensitivity (e.g., faces, scenes), strictly within the context of our task its involvement may be centered on establishing discourse referents. Given the role of the (para)hippocampus in episodic memory, the activity differences we found exclusively for the referential/non-referential contrast may index referent-specific memory traces. The involvement of angular gyrus and anterior IFG in coherence points to a sensitivity to specific Tokens, which cannot be generated from non-referential incoherent sentences.
The sensitivity of pSTS mostly to semantic narrowing but also jointly to referential violations, in addition to its previously documented role in basic semantic composition3 and its greater engagement for inferential naming over picture naming21, points towards an involvement in semantic Types. The clear sensitivity of middle IFS to all semantic processes also points towards an involvement in Types. Both pSTS and IFS are reliably active during semantic unification operations88. Lastly, the sensitivity of OFC to all effects, albeit only during late stages (Supplementary Fig. 4), points towards involvement in Types, with this region being commonly implicated in aspects of semantic saliency and control89,90. We also note that all three of these regions (pSTS, middle IFS, OFC) were not activated early in the sentence (Fig. 1), only towards the end of the sentence. Indeed, while anterior IFS and posterior IFS were both active from words 1 and 2, there was a notable absence of middle IFS activity during early sentence processing, the locus of our mosaic for semantic integration. This suggests again that the effects documented here pertain to higher-order semantic integration processes relevant to type sensitivity. It may be possible to adjudicate here between lexico-semantic engagement (pSTS, IFS, OFC) and domain-general semantic memory activation (dorsal frontal, medial parietal, medial temporal)91,92, in particular, given that PHG, precuneus, and vmPFC are all implicated in general semantic processing93.
We again stress here that our experimental paradigm and analyses were optimized to evince the effects of integrative lexical access and semantic coherence. We have speculated some promising and plausible connections to broader concepts in semantics (Fig. 4) as a means of extending the potential scope of how we might use our results to inform more general concerns in the cognitive neuroscience of language.
The Controlled Semantic Cognition account94,95 is another candidate for helping to frame our results. Under this account, semantic cognition is split into a representation component and a control component, for storing and manipulating semantic information during verbal and non-verbal tasks. Our results implicate many cortical regions involved in representational storage, but also cognitive control regions (i.e., IFS, IFG, pMTG). For example, dorsolateral PFC has shown increased BOLD responses when semantic selection demands are high, whereas ventral PFC and pMTG have shown increased activation during the retrieval of weak semantic associations94. Moreover, activation in the intermediate middle-lateral PFC has been found to correlate with both of the above demands. This model seems to be sympathetic to our discovery of (i) effects of reference in the middle frontal gyrus and IFS, (ii) effects of semantic coherence in pMTG and anterior IFG, and (iii) a region in middle IFS that responds to all semantic demands. We note that our discovery of a cortical mosaic for semantic structure in IFS is sympathetic to the recently uncovered frontal organization of semantic control, which is “sandwiched” between the multiple-demand and default-mode systems96. Our results also help emphasize contributions from medial cortical structures within this Controlled Semantic Cognition framework, such as the MPC.
Overall, despite the limitations of our experimental paradigm, our work suggests that current models apportioning clear frontal, temporal, and parietal separation between semantic integration processes need to accommodate the richly interwoven, mosaic-like architecture of the natural language semantics system3,97,98,99.
We reported extensive whole brain intracranial mapping of semantically coherent orthographic sentence representations, providing insights into semantic integration. We discovered complementary cortical mosaics for semantic integration in the posterior temporal and inferior frontal cortex, recruited for distinct semantic demands. We documented diverse roles for posterior temporal and inferior frontal language regions and dissociated regional contributions to distinct semantic processes. In particular, our intracranial recordings afforded access to sulcal structures as well as the lateral surface of the cortical mantle, with IFS (early sensitivity) and OFC (late sensitivity) uniquely responding to all semantic processes, potentially indexing their roles as higher-order lexico-semantic hubs or sites of computing semantic saliency. Rather than finding posterior temporal and inferior frontal engagement only for successful lexical reference, or only for semantic coherence, we found that partially overlapping sub-regions of these loci are engaged in both (within pSTS, IFS, and aIFG). This topography of functional arrangement implies the existence of small scale regional networks in the lateral frontal and temporal cortex that then connect with the larger scale semantic network distributed across lobes. It remains an open question whether a similar mosaic-like cortical organization underlies the representation of syntactic/grammatical processing and might account for various language dysfunctions100,101. A clearer understanding of these systems will pave the way for deeper insight into developmental dyslexia, and acquired language impairment and potentially enable better neuro-rehabilitative and neuroprosthetic approaches for these disorders102.
Methods
Participants
58 patients (11 male, 18–41 ± 5.7 years, 2 left-handed) participated in the experiment after written informed consent was obtained. All were native English speakers with no history of language deficits. Participants with significant additional neurological history (e.g., previous resections, MR imaging abnormalities such as malformations or hypoplasia, or those with prosopagnosia) were excluded. All patients were subject to full neuropsychological assessments (FSIQ scores: 94.3 ± 13.4). All experimental procedures were reviewed and approved by the Committee for the Protection of Human Subjects (CPHS) of the University of Texas Health Science Center at Houston as Protocol Number: HSC-MS-06-0385. Given that electrode placement in these participants was for clinical need rather than experimental purposes, no statistical methods were used to pre-determine sample sizes, but the number of participants required was based on providing adequate coverage of the language areas being studied.
Electrode implantation and data recording
Data were acquired from either stereotactically placed depth electrodes (sEEGs; 56 patients) or subdural grid electrodes (SDEs; 2 patients). SDEs were subdural platinum-iridium electrodes embedded in a silicone elastomer sheet (PMT Corporation; top-hat design; 3 mm diameter cortical contact) and were surgically implanted via a craniotomy34,103,104,105. sEEGs were implanted using a Robotic Surgical Assistant (ROSA; Medtech, Montpellier, France)106,107. Each sEEG probe (PMT corporation, Chanhassen, Minnesota) was 0.8 mm in diameter and had 8–16 electrode contacts. Each contact was a platinum-iridium cylinder, 2.0 mm in length and separated from the adjacent contact by 1.5–2.43 mm. Each patient had 12–20 such probes implanted (total electrode count: 11,328; M = 199). Following surgical implantation, electrodes were localized by co-registration of pre-operative anatomical 3 T MRI and post-operative CT scans in AFNI108. Electrode positions were projected onto a cortical surface model generated in FreeSurfer109, and displayed on the cortical surface model for visualization104. Intracranial data were collected during research experiments starting on the first day after electrode implantation for sEEGs and two days after implantation for SDEs. Data were digitized at 2 kHz using the NeuroPort recording system (Blackrock Microsystems, Salt Lake City, Utah), imported into Matlab, initially referenced to the white matter channel used as a reference for the clinical acquisition system, and visually inspected for line noise, artifacts, and epileptic activity. Electrodes with excessive line noise or localized to sites of seizure onset were excluded. Each electrode was re-referenced to the common average of the remaining channels. Trials contaminated by inter-ictal epileptic spikes and trials in which participants responded incorrectly were discarded.
Stimuli and experimental design
Patients were asked to quickly and accurately articulate the names of common objects in response to orthographic descriptions (mean: 6.5 words, range: 3–12 words). Sentences varied in length (3–12 words) to reduce the predictability of the location of the final, definition-determining word. A fixation cross was presented in the center of the screen for 1000 ms, followed by each successive word in the sentence, each appearing for 500 ms, and then a blank screen was presented for 1 s. Patients had 2 seconds to respond. The number of trials per block across the full experiment was as follows: Referential (42), Non-Referential (42). Most patients undertook two blocks (n = 48), with some only completing a single block (n = 10). Stimuli were presented using Psychtoolbox110 on a 15.4” LCD screen positioned at eye-level, 2–3’ from the patient. Words were presented in sentence case, in Arial font with a height of 150 pixels (~2.2° visual angle). For the incoherent Non-Referential trials, these either involved a clear theta-role violation or a violation of lexical selectional requirements, what we termed a strong violation; e.g., “Something you use to teach a land” (‘land’ cannot be an Agent), “A period of time that lasts seven boats” (‘boats’ cannot be a temporal measurement). In contrast, the coherent Non-Referential trials did not exhibit these specific violations, and only yielded a violation effect pertaining to the non-existence of a specific nominal denoting the inferred concept, what we termed a weak violation (e.g., “A person at the circus who makes you commute” generates a coherent meaning but lacks a corresponding lexical entry satisfying the definitional description). We did not determine differences in vividness ratings for referential vs. non-referential trials, which we recognize could be subject to future research. Our full set of stimuli is available to access via OSF (osf.io/es7y3).
To determine the profile of semantic narrowing effects in our stimuli, we conducted a norming study on a non-clinical population (n = 80, 50 female, mean age: 34, range: 18–71). All were native English speakers and had a 90% minimum approval rating on Prolific Academic from which they were recruited to take part in the questionnaire (run via the Qualtrics online platform). Participants were paid 10 USD per hour, and the average completion time was 26 min. Participants were presented with the referential trials from the main experiment, word-by-word, and were asked to type any possible corresponding words that might match the ongoing description upon the presentation of a new word. For example, they were first shown “A”, followed by “A round”, then “A round red”, and finally “A round red fruit”. Most participants answered “apple” after being presented with “A round red”, with many answering “ball” or “circle” after seeing “A round”.
Signal analysis
A total of 13,298 electrode contacts were implanted in these patients; 9388 of these were included for analysis after excluding channels proximal to the seizure onset zone or exhibiting excessive inter-ictal spikes or line noise. Analyses were performed by first bandpass filtering the raw data of each electrode into broadband gamma activity (BGA; 70–150 Hz) following removal of line noise and its harmonics (zero-phase 2nd order Butterworth band-stop filters)111. A frequency domain bandpass Hilbert transform (paired sigmoid flanks with half-width 1.5 Hz) was applied and the analytic amplitude was smoothed (Savitzky-Golay FIR, 3rd order, frame length of 251 ms; Matlab 2019b, Mathworks, Natick, MA). BGA was defined as a percentage change from baseline level; −500 ms to −100 ms before the presentation of the first word in each trial. Periods of significant activation were tested using a one-tailed t-test at each time point and were corrected for multiple comparisons with a Benjamini-Hochberg false detection rate (FDR) corrected threshold of q < 0.05. For the grouped analysis (across multiple patients), all electrodes were averaged within each subject and then the between-subject averages were used.
To provide statistically robust and topologically precise estimates of BGA, and to account for variations in sampling density, population-level representations were created using surface-based mixed-effects multilevel analysis (SB-MEMA)34,40,41,42. We focus our analyses on the language-dominant left hemisphere (but see also Supplementary Fig. 3 for right hemisphere responses to reference). This method accounts for sparse sampling, outlier inferences, as well as intra- and inter-subject variability to produce population maps of cortical activity. A geodesic Gaussian smoothing filter (3 mm full-width at half-maximum) was applied. The minimum criterion for the family-wise error rate was determined by white-noise clustering analysis (Monte Carlo simulations, 5000 iterations) of data with the same dimension and smoothness as that analyzed. Results were further restricted to regions with at least three patients contributing to coverage. Due to the proximity to loci with typical eye movement artefacts112, we also evaluated right-hemispheric ventromedial prefrontal cortex and verified that reported effects were specific to the left hemisphere (Supplementary Figs. 1, 3). Likewise, we confirmed that only the language-dominant left hemisphere medial structures, and not right hemisphere structures, index sensitivity to referential sentences. Lastly, the conjunction SB-MEMA maps (Fig. 4) depict regions sensitive to conditional contrasts, regardless of directionality. The individual MEMA contrast maps were binarized (threshold: %BGA > 5%, t > 2.57, patient coverage ≥2) and the logical Boolean values for these three maps were computed to plot which regions showed significant activity differences for each combination (e.g., for narrowing and coherence, but not reference).
Our regions of interest were clustered from a number of Human Connectome Project (HCP) regions113: Medial parietal cortex (RSC, POS1, POS2, v23ab, 7 m, 31pv, 31pd, d23ab); hippocampus and parahippocampal gyrus (PreS, EC, PHA1); ventromedial prefrontal cortex (a24, p32, 10r, s32, 10 v, 25, OFC); orbitofrontal cortex (11 l, 13 l); middle frontal gyrus (8 C, 46, p9, 46 v); dorsal inferior frontal gyrus and inferior frontal sulcus (IFSa); posterior superior temporal sulcus (TPOJ1, TPOJ2). For the low frequency analysis in the hippocampus, we manually checked electrode placements to ensure a distinction between electrodes placed within the hippocampus proper and within neighboring parahippocampal sites (Parahippocampus = PreSubiculum/Entorhinal cortex/Parahippocampal Area 1).
The language system is richly interwoven with episodic memory networks, with segregation between these systems resulting more from investigator domain expertise than neurobiological underpinnigs. Given that hippocampal theta is modulated by sentence predictability70, theta phase-coupling occurs between the hippocampus and left superior temporal gyrus increases during correct (vs. incorrect) sentences114, and theta power in the inferior frontal cortex and central EEG sites is linked to language comprehension115,116, we also examined low frequency dynamics across these sites.
Anatomical groups of electrodes were delineated, firstly, through indexing electrodes to the closest node on the standardized cortical surface117, and secondly, through grouping channels into parcellations determined by Human Connectome Project (HCP) space113. Parametric statistics were used since HCP regions of interest contained >30 electrodes. To determine significant activity increases from baseline, a two-sided paired t-test was evaluated at each time point for each region and significance levels were computed at a corrected alpha-level of 0.05 using an FDR correction for multiple comparisons. To determine a significant difference between conditions in the activation of a region, the cumulative BGA in a specified time window was evaluated with a two-sided paired t-test at an alpha-level of 0.01.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
The datasets generated from this research are not publicly available due to them containing information non-compliant with HIPAA and the human participants the data were collected from have not consented to their public release. However, they are available on request from the corresponding author. Our full set of stimuli is available to access (osf.io/es7y3). Source data are provided with this paper.
Code availability
The custom code that supports the findings of this study is available from the corresponding author on request.
References
Baggio, G. Meaning in the Brain. (MIT Press, 2018).
Friederici, A. D. Language in our brain: the origins of a uniquely human capacity. (The MIT Press, 2017):https://doi.org/10.7551/mitpress/9780262036924.001.0001.
Murphy, E. et al. Minimal phrase composition revealed by intracranial recordings. J. Neurosci. 42, 3216–3227 (2022).
Burge, T. Reference and proper names. J. Philos. 70, 425 (1973).
Chomsky, N. New Horizons in the study of language and mind. new horizons in the study of language and mind (Cambridge University Press, 2000). https://doi.org/10.1017/cbo9780511811937.
Davidson, D. Reality without reference. Dialectica 31, 247–258 (1977).
Kripke, S. Naming and necessity. (Harvard University Press, 1980).
Kripke, S. A. Reference and existence: the John Locke lectures. (Oxford University Press, 2013). https://doi.org/10.1093/acprof:oso/9780199928385.001.0001.
Murphy, E. The oscillatory nature of language. (Cambridge University Press, 2020). https://doi.org/10.1017/9781108864466.
Pietroski, P. M. Conjoining meanings. vol. 1 (Oxford University Press, 2018).
Russell, B. On denoting. Mind XIV, 479–493 (1905).
Strawson, P. F. On referring. Mind1 59, 320–344 (1950).
Murphy, E. & Leivada, E. A model for learning strings is not a model of language. Proc. Natl Acad. Sci. U.S.A. 119, e2201651119 (2022).
Berwick, R. C. & Stabler, E. P. Minimalist Parsing. (Oxford University Press, 2019). https://doi.org/10.1093/oso/9780198795087.001.0001.
Matchin, W. & Hickok, G. The cortical organization of syntax. Cereb. Cortex 30, 1481–1498 (2020).
Brennan, J. R., Stabler, E. P., Van Wagenen, S. E., Luh, W. M. & Hale, J. T. Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain Lang. 157–158, 81–94 (2016).
McCarty, M. J. et al. Intraoperative cortical localization of music and language reveals signatures of structural complexity in posterior temporal cortex. iScience 26, 107223 (2023).
Nieuwland, M. S., Petersson, K. M. & Van Berkum, J. J. A. On sense and reference: examining the functional neuroanatomy of referential processing. NeuroImage 37, 993–1004 (2007).
Ferstl, E. C., Neumann, J., Bogler, C. & Von Cramon, D. Y. The extended language network: a meta-analysis of neuroimaging studies on text comprehension. Human Brain Mapping (2008) https://doi.org/10.1002/hbm.20422.
Brodbeck, C. & Pylkkänen, L. Language in context: Characterizing the comprehension of referential expressions with MEG. NeuroImage 147, 447–460 (2017).
Marconi, D. et al. The neural substrates of inferential and referential semantic processing. Cortex 49, 2055–2066 (2013).
Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K. & Sedivy, J. Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632–1634 (1995).
Huettig, F., Rommers, J. & Meyer, A. S. Using the visual world paradigm to study language processing: a review and critical evaluation. Acta Psychol. 137, 151–171 (2011).
Camblin, C. C., Gordon, P. C. & Swaab, T. Y. The interplay of discourse congruence and lexical association during sentence processing: evidence from ERPs and eye tracking. J. Mem. Lang. 56, 103–128 (2007).
Jouen, A.-L. et al. Common ERP responses to narrative incoherence in sentence and picture pair comprehension. Brain Cognition 153, 105775 (2021).
Ivanova, A. A. et al. The language network is recruited but not required for nonverbal event semantics. Neurobiol. Lang. 2, 176–201 (2021).
Sueoka, Y., Paunov, A., Ivanova, A., Blank, I. A. & Fedorenko, E. The language network reliably ‘tracks’ naturalistic meaningful non-verbal stimuli. http://biorxiv.org/lookup/doi/10.1101/2022.04.24.489316, https://doi.org/10.1101/2022.04.24.489316 (2022).
Colvin, M., Warren, T. & Dickey, M. W. Event knowledge and verb knowledge predict sensitivity to different aspects of semantic anomalies in Aphasia. in Grammatical Approaches to Language Processing (eds. Carlson, K., Clifton, C. & Fodor, J. D.) vol. 48 241–259 (Springer International Publishing, 2019).
Schell, M., Zaccarella, E. & Friederici, A. D. Differential cortical contribution of syntax and semantics: an fMRI study on two-word phrasal processing. Cortex 96, 105–120 (2017).
van der Burght, C. L., Numssen, O., Schlaak, B., Goucha, T. & Hartwigsen, G. Differential contributions of inferior frontal gyrus subregions to sentence processing guided by intonation. Hum. Brain Mapp. 44, 585–598 (2023).
Schwen Blackett, D. et al. Neural network bases of thematic semantic processing in language production. Cortex 156, 126–143 (2022).
Arya, R. Similarity of spatiotemporal dynamics of language-related ECoG high-gamma modulation in Japanese and English speakers. Clin. Neurophysiol. 130, 1403–1404 (2019).
Flinker, A., Chang, E. F., Barbaro, N. M., Berger, M. S. & Knight, R. T. Sub-centimeter language organization in the human temporal lobe. Brain Lang. 117, 103–109 (2011).
Conner, C. R., Ellmore, T. M., Pieters, T. A., di Sano, M. A. & Tandon, N. Variability of the relationship between electrophysiology and BOLD-fMRI across cortical regions in humans. J. Neurosci. 31, 12855–12865 (2011).
Penolazzi, B., Angrilli, A. & Job, R. Gamma EEG activity induced by semantic violation during sentence reading. Neurosci. Lett. 465, 74–78 (2009).
Obleser, J. & Kotz, S. A. Multiple brain signatures of integration in the comprehension of degraded speech. NeuroImage 55, 713–723 (2011).
Wang, Y., Korzeniewska, A., Usami, K., Valenzuela, A. & Crone, N. E. The dynamics of language network interactions in lexical selection: an intracranial EEG study. Cereb. Cortex 31, 2058–2070 (2021).
Staub, A., Grant, M., Astheimer, L. & Cohen, A. The influence of cloze probability and item constraint on cloze task response time. J. Mem. Lang. 82, 1–17 (2015).
Kutas, M. & Hillyard, S. A. Brain potentials during reading reflect word expectancy and semantic association. Nature 307, 161–163 (1984).
Fischl, B., Sereno, M. I. & Dale, A. M. Cortical surface-based analysis: II. Inflation, flattening, and a surface-based coordinate system. NeuroImage 9, 195–207 (1999).
Kadipasaoglu, C. M. et al. Surface-based mixed effects multilevel analysis of grouped human electrocorticography. NeuroImage 101, 215–224 (2014).
Kadipasaoglu, C. M. et al. Development of grouped icEEG for the study of cognitive processing. Front. Psychol. 6, 1008 (2015).
Woolnough, O. et al. Spatiotemporal dynamics of orthographic and lexical processing in the ventral visual pathway. Nat. Hum. Behav. 5, 389–398 (2021).
Hagoort, P. & Indefrey, P. The neurobiology of language beyond single words. Annu. Rev. Neurosci. 37, 347–362 (2014).
Kuhnke, P., Meyer, L., Friederici, A. D. & Hartwigsen, G. Left posterior inferior frontal gyrus is causally involved in reordering during sentence processing. NeuroImage 148, 254–263 (2017).
Price, C. J. The anatomy of language: a review of 100 fMRI studies published in 2009. Ann. N. Y. Acad. Sci. 1191, 62–88 (2010).
Halgren, E. et al. N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences. NeuroImage 17, 1101–1116 (2002).
Conner, C. R., Chen, G., Pieters, T. A. & Tandon, N. Category specific spatial dissociations of parallel processes underlying visual naming. Cereb. Cortex 24, 2741–2750 (2014).
Forseth, K. J. et al. A lexical semantic hub for heteromodal naming in middle fusiform gyrus. Brain 141, 2112–2126 (2018).
Swanson, S. J. et al. Changes in description naming for common and proper nouns after left anterior temporal lobectomy. Epilepsy Behav. 106, 106912 (2020).
Snyder, K. M. et al. Critical role of the ventral temporal lobe in naming. Epilepsia 64, 1200–1213 (2023).
Roos, N. M., Takashima, A. & Piai, V. Functional neuroanatomy of lexical access in contextually and visually guided spoken word production. Cortex 159, 254–267 (2023).
Spalding, K. N. et al. Ventromedial prefrontal cortex is necessary for normal associative inference and memory integration. J. Neurosci. 38, 3767–3775 (2018).
Brodbeck, C., Gwilliams, L. & Pylkkänen, L. Language in context: MEG evidence for modality-general and -specific responses to reference resolution. eNeuro https://doi.org/10.1523/ENEURO.0145-16.2016 (2016).
Buckner, R. L. & DiNicola, L. M. The brain’s default network: updated anatomy, physiology and evolving insights. Nat. Rev. Neurosci. 20, 593–608 (2019).
Alexander, A. S., Place, R., Starrett, M. J., Chrastil, E. R. & Nitz, D. A. Rethinking retrosplenial cortex: perspectives and predictions. Neuron 111, 150–175 (2023).
Willbrand, E. H. et al. Sulcal morphology of posteromedial cortex substantially differs between humans and chimpanzees. Preprint at https://doi.org/10.1101/2023.02.06.527223 (2023).
Duncan, J., Assem, M. & Shashidhara, S. Integrated intelligence from distributed brain activity. Trends Cogn. Sci. 24, 838–852 (2020).
Fedorenko, E., Duncan, J. & Kanwisher, N. Broad domain generality in focal regions of frontal and parietal cortex. Proc. Natl Acad. Sci. USA 110, 16616–16621 (2013).
Boiteau, T. W., Bowers, E., Nair, V. A. & Almor, A. The neural representation of plural discourse entities. Brain and Language https://doi.org/10.1016/j.bandl.2014.08.003 (2014).
Ranganath, C. & Ritchey, M. Two cortical systems for memory-guided behaviour. Nat. Rev. Neurosci. 13, 713–726 (2012).
Lundstrom, B. N., Ingvar, M. & Petersson, K. M. The role of precuneus and left inferior frontal cortex during source memory episodic retrieval. NeuroImage https://doi.org/10.1016/j.neuroimage.2005.05.008 (2005).
Rabini, G., Ubaldi, S. & Fairhall, S. L. Combining concepts across categorical domains: a linking role of the precuneus. Neurobiol. Lang. 1–18 https://doi.org/10.1162/nol_a_00039 (2021).
Rajah, M. N. & McIntosh, A. R. Overlap in the functional neural systems involved in semantic and episodic memory retrieval. J. Cogn. Neurosci. 17, 470–482 (2005).
Schacter, D. L. & Addis, D. R. The cognitive neuroscience of constructive memory: remembering the past and imagining the future. in Philosophical Transactions of the Royal Society B: Biological Sciences https://doi.org/10.1098/rstb.2007.2087 (2007).
Zwaan, R. A. Situation models, mental simulations, and abstract concepts in discourse comprehension. Psychonomic Bulletin and Review Preprint at https://doi.org/10.3758/s13423-015-0864-x (2016).
Silson, E. H., Steel, A., Kidder, A., Gilmore, A. W. & Baker, C. I. Distinct subdivisions of human medial parietal cortex support recollection of people and places. eLife 8, e47391 (2019).
Woolnough, O. et al. Category selectivity for face and scene recognition in human medial parietal cortex. Curr. Biol. 30, 2707–2715.e3 (2020).
Hebscher, M., Meltzer, J. A. & Gilboa, A. A causal role for the precuneus in network-wide theta and gamma oscillatory activity during complex memory retrieval. eLife 8, e43114 (2019).
Piai, V. et al. Direct brain recordings reveal hippocampal rhythm underpinnings of language processing. Proc. Natl Acad. Sci. USA 113, 11366–11371 (2016).
Boran, E. et al. Dataset of human medial temporal lobe neurons, scalp and intracranial EEG during a verbal working memory task. Scientific Data https://doi.org/10.1038/s41597-020-0364-3 (2020).
Aminoff, E. M., Kveraga, K. & Bar, M. The role of the parahippocampal cortex in cognition. Trends Cogn. Sci. 17, 379–390 (2013).
Benítez-Burraco, A. & Murphy, E. Why brain oscillations are improving our understanding of language. Front. Behav. Neurosci. 13, 190 (2019).
Murphy, E., Hoshi, K. & Benítez-Burraco, A. Subcortical syntax: reconsidering the neural dynamics of language. J. Neurolinguist. 62, 101062 (2022).
Klooster, N. B. & Duff, M. C. Remote semantic memory is impoverished in hippocampal amnesia. Neuropsychologia https://doi.org/10.1016/j.neuropsychologia.2015.10.017 (2015).
Lega, B. C., Jacobs, J. & Kahana, M. Human hippocampal theta oscillations and the formation of episodic memories. Hippocampus 22, 748–761 (2012).
Benítez-Burraco, A. Mental time travel, language evolution, and human self-domestication. Cognitive Processing https://doi.org/10.1007/s10339-020-01005-2 (2021).
Maurer, A. P. & Nadel, L. The continuity of context: a role for the hippocampus. Trends Cogn. Sci. 25, 187–199 (2021).
Hickok, G., Venezia, J. & Teghipco, A. Beyond Broca: neural architecture and evolution of a dual motor speech coordination system. Brain https://doi.org/10.1093/brain/awac454 (2022).
Friederici, A. D. The Brain Basis of Language Processing: From Structure to Function. Physiol. Rev. 91, 1357–1392 (2011).
Branco, P., Seixas, D. & Castro, S. L. Mapping language with resting‐state functional magnetic resonance imaging: a study on the functional profile of the language network. Hum. Brain Mapp. 41, 545–560 (2020).
Matchin, W., Brodbeck, C., Hammerly, C. & Lau, E. The temporal dynamics of structure and content in sentence comprehension: evidence from fMRI-constrained MEG. Hum. Brain Mapp. 40, 663–678 (2019).
Binder, J. R. & Desai, R. H. The neurobiology of semantic memory. Trends Cogn. Sci. 15, 527–536 (2011).
Flick, G. & Pylkkänen, L. Isolating syntax in natural language: MEG evidence for an early contribution of left posterior temporal cortex. Cortex 127, 42–57 (2020).
Gow, D. W. The cortical organization of lexical knowledge: a dual lexicon model of spoken language processing. Brain Lang. 121, 273–288 (2012).
Thye, M., Geller, J., Szaflarski, J. P. & Mirman, D. Intracranial EEG evidence of functional specialization for taxonomic and thematic relations. Cortex 140, 40–50 (2021).
Forseth, K. J., Pitkow, X., Fischer-Baum, S. & Tandon, N. What the brain does as we speak. Preprint at https://doi.org/10.1101/2021.02.05.429841 (2021).
Zhu, Z. et al. Semantic unification modulates N400 and BOLD signal change in the brain: a simultaneous EEG-fMRI study. J. Neurolinguist. 52, 100855 (2019).
Hertrich, I., Dietrich, S. & Ackermann, H. The margins of the language network in the brain. Front. Commun. 5, 519955 (2020).
Ding, J. et al. A unified neurocognitive model of semantics language social behaviour and face recognition in semantic dementia. Nat. Commun. 11, 2595 (2020).
Binder, J. R. & Fernandino, L. Semantic Processing. in Brain Mapping 445–454 (Elsevier, 2015). https://doi.org/10.1016/B978-0-12-397025-1.00266-9.
Hagoort, P. The meaning-making mechanism(s) behind the eyes and between the ears. Philos. Trans. R. Soc. B 375, 20190301 (2020).
Binder, J. R., Desai, R. H., Graves, W. W. & Conant, L. L. Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796 (2009).
Ralph, M. A. L., Jefferies, E., Patterson, K. & Rogers, T. T. The neural and computational bases of semantic cognition. Nat. Rev. Neurosci. 18, 42–55 (2017).
Rogers, T. T. & McClelland, J. L. Semantic Cognition: A Parallel Distributed Processing Approach. (The MIT Press, 2004). https://doi.org/10.7551/mitpress/6161.001.0001.
Chiou, R., Jefferies, E., Duncan, J., Humphreys, G. F. & Lambon Ralph, M. A. A middle ground where executive control meets semantics: the neural substrates of semantic control are topographically sandwiched between the multiple-demand and default-mode systems. Cerebral Cortex bhac358 https://doi.org/10.1093/cercor/bhac358 (2022).
Murphy, E. The brain dynamics of linguistic computation. Front. Psychol. 6, 1515 (2015).
Woolnough, O. et al. Spatiotemporally distributed frontotemporal networks for sentence reading. Proc. Natl Acad. Sci. USA 120, e2300252120 (2023).
Murphy, E. ROSE: A neurocomputational architecture for syntax. Preprint at http://arxiv.org/abs/2303.08877 (2023).
Benítez-Burraco, A., Hoshi, K. & Murphy, E. Language deficits in GRIN2A mutations and Landau–Kleffner syndrome as neural dysrhythmias. J. Neurolinguist. 67, 101139 (2023).
Murphy, E. & Benítez-Burraco, A. Language deficits in schizophrenia and autism as related oscillatory connectomopathies: an evolutionary account. Neurosci. Biobehav. Rev. 83, 742–764 (2017).
Thomas, T. M. et al. Decoding articulatory and phonetic components of naturalistic continuous speech from the distributed language network. J. Neural Eng. 20, 046030 (2023).
Tandon, N. Mapping of human language. in Clinical Brain Mapping (eds. Yoshor, D. & Mizrahi, E.) 203–218 (McGraw Hill Education, 2012).
Pieters, T. A., Conner, C. R. & Tandon, N. Recursive grid partitioning on a cortical surface model: an optimized technique for the localization of implanted subdural electrodes. J. Neurosurg. 118, 1086–1097 (2013).
Tong, B. A., Esquenazi, Y., Johnson, J., Zhu, P. & Tandon, N. The brain is not flat: conformal electrode arrays diminish complications of subdural electrode implantation, a series of 117 cases. World Neurosurg. 144, e734–e742 (2020).
Tandon, N. et al. Analysis of morbidity and outcomes associated with use of subdural grids vs stereoelectroencephalography in patients with intractable epilepsy. JAMA Neurol. 76, 672–681 (2019).
Rollo, P. S., Rollo, M. J., Zhu, P., Woolnough, O. & Tandon, N. Oblique trajectory angles in robotic stereo-electroencephalography. J. Neurosurg. 1–10 https://doi.org/10.3171/2020.5.JNS20975 (2020).
Cox, R. W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173 (1996).
Dale, A. M., Fischl, B. & Sereno, M. I. Cortical surface-based analysis: i. segmentation and surface reconstruction. NeuroImage 9, 179–194 (1999).
Kleiner, M. et al. What’s new in psychtoolbox-3. Perception 36, 1–16 (2007).
McCarty, M. J., Woolnough, O., Mosher, J. C., Seymour, J. & Tandon, N. The Listening Zone of Human Electrocorticographic Field Potential Recordings. eNeuro ENEURO.0492-21.2022 https://doi.org/10.1523/ENEURO.0492-21.2022 (2022).
Kovach, C. K. et al. Manifestation of ocular-muscle EMG contamination in human intracranial recordings. NeuroImage 54, 213–233 (2011).
Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016).
Pu, Y., Cheyne, D., Sun, Y. & Johnson, B. W. Theta oscillations support the interface between language and memory. NeuroImage 215, 116782 (2020).
Mellem, M. S., Friedman, R. B. & Medvedev, A. V. Gamma- and theta-band synchronization during semantic priming reflect local and long-range lexical–semantic networks. Brain Lang. 127, 440–451 (2013).
Momsen, J. P. & Abel, A. D. Neural oscillations reflect meaning identification for novel words in context. Neurobiology of Language 1–40 https://doi.org/10.1162/nol_a_00052 (2021).
Saad, Z. S. & Reynolds, R. C. SUMA. NeuroImage 62, 768–773 (2012).
Acknowledgements
We express our gratitude to all the patients who participated in this study; the neurologists at the Texas Comprehensive Epilepsy Program who participated in the care of these patients; and the nurses and technicians in the Epilepsy Monitoring Unit at Memorial Hermann Hospital who helped make this research possible. We also sincerely thank the three reviewers for providing insightful and thoughtful comments that greatly improved the presentation of our work. This work was supported by the National Institute of Neurological Disorders and Stroke NS098981. Human subjects: Patients participated in the experiments after written informed consent was obtained. All experimental procedures were reviewed and approved by the Committee for the Protection of Human Subjects (CPHS) of the University of Texas Health Science Center at Houston as Protocol Number: HSC-MS-06-0385.
Author information
Authors and Affiliations
Contributions
Conceptualization: E.M., K.J.F., C.D., N.T.; Methodology, Writing – Review and Editing: E.M., K.J.F., N.T.; Data curation: E.M., K.J.F., C.D., K.S., P.S.R.; Software, Formal Analysis and Visualization: E.M., K.J.F., K.S.; Writing – Original Draft: E.M.; E.M., K.J.F., N.T.; Funding Acquisition, Neurosurgical Procedures: NT.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks William Matchin and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Source data
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Murphy, E., Forseth, K.J., Donos, C. et al. The spatiotemporal dynamics of semantic integration in the human brain. Nat Commun 14, 6336 (2023). https://doi.org/10.1038/s41467-023-42087-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-023-42087-8
- Springer Nature Limited
This article is cited by
-
The speech neuroprosthesis
Nature Reviews Neuroscience (2024)
-
Natural language syntax complies with the free-energy principle
Synthese (2024)