Abstract
When listeners follow spoken instructions to manipulate real objects, their eye movements to the objects are closely time locked to the referring words. We review five experiments showing that this time-locked characteristic of eye movements provides a detailed profile of the processes that underlie real-time spoken language comprehension. Together, the first four experiments showed that listerners immediately integrated lexical, sublexical, and prosodic information in the spoken input with information from the visual context to reduce the set of referents to the intended one. The fifth experiment demonstrated that a visual referential context affected the initial structuring of the linguistic input, eliminating even strong syntactic preferences that result in clear garden paths when the referential context is introduced linguistically. We argue that context affected the earliest moments of language processing because it was highly accessible and relevant to the behavioral goals of the listener.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abney, S. (1989). A computational model of human parsing.Journal of Psycholinguistic Research, 18, 129–144.
Altmann, G. (1987). Modularity and interaction in sentence processing. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp 249–257). Cambridge: MIT Press.
Altmann, G., & Steedman, M. (1988). Interaction with context during human sentence processing,Cognition, 30, 191–238.
Bates, E., & MacWhinney, B. (1989). Functionalism and the competition model. In B. MacWhinney & E. Bates (Eds.),The crosslinguistic study of sentence processing. New York: Cambridge University Press.
Boland, J., Tanenhaus, M. K., Garnsey, S., & Carlson, G. (in press). Argument structure and filler-gap assignment.Journal of Memory and Language.
Britt, M. A. (1994). The interaction of referential ambiguity and argument structure in the parsing of prepositional phrases.Journal of Memory and Language, 33, 251–283.
Clark, H. H. (1992).Arenas of language use. Chicago: University of Chicago Press.
Clark, H. H., & Marshall, C. R. (1992). Definite reference and mutual knowledge. In H. H. Clark, (Ed.),Arenas of language use (pp. 9–59). Chicago: University of Chicago Press.
Crain, S., & Steedman, M. (1985). On not being led up the garden path: The use of context by the psychological parser. In D. Dowty, L. Kartunnen, & H. Zwicky (Eds.),Natural language parsing. Cambridge, England: Cambridge University Press.
Eberhard, K., Tanenhaus, M., Spivey-Knowlton, M., & Sedivy, J. (1995).Investigating the time course of establishing reference: Evidence for rapid incremental processing of spoken language. Unpublished manuscript.
Ferreira, F., & Clifton, C. (1986). The independence of syntactic processing.Journal of Memory and Language, 25, 348–368.
Frazier, L. (1978).On comprehending sentences: Syntactic parsing strategies. Unpublished doctoral dissertation, University of Connecticut, Storrs.
Frazier, L. (1987). Sentence processing: A tutorial review. In M. Coltheart (Ed.),Attention and performance XII. Hove, England: Erlbaum.
Grice (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.),Syntax and semantics: Vol. 3. Speech acts. New York: Academic Press.
Jackendoff, R. (1972).Semantic interpretation in generative grammar. Cambridge, MA: MIT Press.
Kanerva (1990). Focusing on phonological phrases in Chichewa. In S. Inkelas & D. Zec (Eds.),The phonology-syntax connection (pp. 145–161). Chicago: University of Chicago Press.
Krifka, M. (1991). A compositional semantics for multiple focus constructions.Proceedings of Semantics and Linguistic Theory (SALT) 1 (Cornell University Working Papers 11.)
MacDonald, M., Pearlmutter, N., & Seidenberg, M. (1994). Lexical nature of syntatic ambiguity resolution.Psychological Review, 101, 676–703.
Marslen-Wilson, W., & Tyler, K. (1987). Against modularity. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp. 37–62). Cambridge, MA: MIT Press.
Marslen-Wilson, W. D. (1987). Functional parallelism in spoken work recognition.Cognition, 25, 71–102.
Matin, E., Shao, K. C., & Boff, K. R. (1993). Saccadic overhead: Information processing time with and without saccades.Perception & Psychophysics, 53, 372–380.
Mitchell, D. C., Corley, M. M. B., & Garnham, A. (1992). Effects of context in human sentence parsing: Evidence against a discourse-based proposal mechanism.Journal of Experimental Psychology: Learning, Memory and Cognition, 18, 69–88.
Olson, D. (1970). Language and thought: Aspects of cognitive theory of semantics.Psychological Review, 77, 143–184.
Perfetti, C. A. (1990). The cooperative language processors: Semantic influences in an autonomous syntax. In D. A. Balota, G. B. Flores d'Arcais, & K. Rayner (Eds.),Comprehension processes in reading. Hillsdale, NJ: Erlbaum.
Pritchett, B. L. (1992).Grammatical competence and parsing performance. Chicago: University of Chicago Press.
Rooth, M. (1985).Association with focus. Unpublished doctoral dissertation, University of Massachusetts, Amherst.
Rooth, M. (1992). A theory of focus interpretation.Natural Language Semantics, 1, 75–116.
Sedivy, J., Carlson, G., Tanenhaus, M., Spivey-Knowlton, M., & Eberhard, K. (1994). The cognitive function of contrast sets in processing focus constructions. InWorking Papers of the IBM Institute for Logic and Linguistics.
Sedivy, J., Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Carlson, G. (1995). Using intonationally-marked presuppositional information in on-line language processing: Evidence from eye movements to a visual model.Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 375–380). Mahwah, NJ: LEA Associates.
Spivey-Knowlton, M., & Sedivy, J. (1995). Parsing attachment ambiguities with multiple constraints.Cognition, 55, 227–267.
Spivey-Knowlton, M., Sedivy, J., Eberhard, K., & Tanenhaus, M. (1994). Psycholinguistic study of the interaction between language and vision. InProceedings of the 12th National Conference on Artificial Intelligence: Workshop on the Integration of Natural Language and Vision Processing.
Spivey-Knowlton, M., & Tanenhaus, M. K. (1994). Referential context and syntactic ambiguity resolution. In C. Clifton, L. Frazier, & K. Rayner (Eds.),Perspectives on sentence processing. Hillsdale, NJ: Erlbaum.
Spivey-Knowlton, M., & Tanenhaus, M. (1995).Syntactic ambiguity resolution in discourse: Modeling the effects of referential context and lexical frequency within an integration-competition framework. Manuscript submitted for publication.
Spivey-Knowlton, M., Tanenhaus, M., Eberhard, K., & Sedivy, J. (1995). Eye-movements accompanying language and action in a visual context: Evidence against modularity. (Eds.),Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 25–30). Mahwah, NJ: LEA Associates.
Spivey-Knowlton, M., Trueswell, J., & Tanenhaus, M. (1993). Context effects in syntactic ambiguity resolution: Discourse and semantic influences in parsing reduced relative clauses.Canadian Journal of Experimental Psychology, 37, 276–309.
Steedman, M. (1987). Combinatory grammars and human language processing. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp. 187–205). Cambridge, MA: MIT Press.
Swinney, D., & Osterhout, L. (1990). Interference generation during auditory language comprehension.The Psychology of Learning and Motivation, 25, 17–33.
Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (1995). The interaction of visual and linguistic information in spoken language comprehension.Science, 268, 1632–1634.
Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (in press). Using eye-movements to study spoken language comprehension: Evidence for visually-mediated incremental interpretation. In T. Inui & J. McClelland (Eds.),Attention & Performance XVI: Integration in Perception and Communication.
Tanenhaus, M., & Trueswell, J. (1995). Sentence comprehension. In J. Miller & P. Eimas (Eds.),Handbook of cognition and perception: Vol. 11. Speech and Language. San Diego, CA: Academic Press.
Taraban, R., & McClelland, J. (1988). Constituent attachment and thematic role expectations.Journal of Memory and Language, 27, 597–632.
Taraban, R., & McClelland, J. (1990). Sentence comprehension: A multiple constraints view. In D. Balota, K. Rayner, & G. Flores d'Arcais (Eds.),Comprehension processes in reading. Hillsdale, NJ: Erlbaum.
Author information
Authors and Affiliations
Additional information
We thank D. Ballard and M. Hayhoe for the use of their laboratory (National Resource Laboratory for the Study of Brain and Behavior). We also thank J. Pelz for his assistance in learning how to use the equipment and K. Kobashi for assisting in the data collection. Finally, we thank Janet Nicol and an anonymous reviewer for their comments and suggestions. The research was supported by NIH resource grant 1-P41-RR09283; NIH HD27206 (M.K.T.); an NSF graduate fellowship (M.J.S.-K.); and a Canadian SSHRC fellowship (J.C.S.).
Rights and permissions
About this article
Cite this article
Eberhard, K.M., Spivey-Knowlton, M.J., Sedivy, J.C. et al. Eye movements as a window into real-time spoken language comprehension in natural contexts. J Psycholinguist Res 24, 409–436 (1995). https://doi.org/10.1007/BF02143160
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02143160