Journal of Psycholinguistic Research

, Volume 24, Issue 6, pp 409–436 | Cite as

Eye movements as a window into real-time spoken language comprehension in natural contexts

  • Kathleen M. Eberhard
  • Michael J. Spivey-Knowlton
  • Julie C. Sedivy
  • Michael K. Tanenhaus


When listeners follow spoken instructions to manipulate real objects, their eye movements to the objects are closely time locked to the referring words. We review five experiments showing that this time-locked characteristic of eye movements provides a detailed profile of the processes that underlie real-time spoken language comprehension. Together, the first four experiments showed that listerners immediately integrated lexical, sublexical, and prosodic information in the spoken input with information from the visual context to reduce the set of referents to the intended one. The fifth experiment demonstrated that a visual referential context affected the initial structuring of the linguistic input, eliminating even strong syntactic preferences that result in clear garden paths when the referential context is introduced linguistically. We argue that context affected the earliest moments of language processing because it was highly accessible and relevant to the behavioral goals of the listener.


Cognitive Psychology Language Processing Real Object Initial Structure Language Comprehension 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abney, S. (1989). A computational model of human parsing.Journal of Psycholinguistic Research, 18, 129–144.CrossRefGoogle Scholar
  2. Altmann, G. (1987). Modularity and interaction in sentence processing. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp 249–257). Cambridge: MIT Press.Google Scholar
  3. Altmann, G., & Steedman, M. (1988). Interaction with context during human sentence processing,Cognition, 30, 191–238.CrossRefPubMedGoogle Scholar
  4. Bates, E., & MacWhinney, B. (1989). Functionalism and the competition model. In B. MacWhinney & E. Bates (Eds.),The crosslinguistic study of sentence processing. New York: Cambridge University Press.Google Scholar
  5. Boland, J., Tanenhaus, M. K., Garnsey, S., & Carlson, G. (in press). Argument structure and filler-gap assignment.Journal of Memory and Language.Google Scholar
  6. Britt, M. A. (1994). The interaction of referential ambiguity and argument structure in the parsing of prepositional phrases.Journal of Memory and Language, 33, 251–283.CrossRefGoogle Scholar
  7. Clark, H. H. (1992).Arenas of language use. Chicago: University of Chicago Press.Google Scholar
  8. Clark, H. H., & Marshall, C. R. (1992). Definite reference and mutual knowledge. In H. H. Clark, (Ed.),Arenas of language use (pp. 9–59). Chicago: University of Chicago Press.Google Scholar
  9. Crain, S., & Steedman, M. (1985). On not being led up the garden path: The use of context by the psychological parser. In D. Dowty, L. Kartunnen, & H. Zwicky (Eds.),Natural language parsing. Cambridge, England: Cambridge University Press.Google Scholar
  10. Eberhard, K., Tanenhaus, M., Spivey-Knowlton, M., & Sedivy, J. (1995).Investigating the time course of establishing reference: Evidence for rapid incremental processing of spoken language. Unpublished manuscript.Google Scholar
  11. Ferreira, F., & Clifton, C. (1986). The independence of syntactic processing.Journal of Memory and Language, 25, 348–368.CrossRefGoogle Scholar
  12. Frazier, L. (1978).On comprehending sentences: Syntactic parsing strategies. Unpublished doctoral dissertation, University of Connecticut, Storrs.Google Scholar
  13. Frazier, L. (1987). Sentence processing: A tutorial review. In M. Coltheart (Ed.),Attention and performance XII. Hove, England: Erlbaum.Google Scholar
  14. Grice (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.),Syntax and semantics: Vol. 3. Speech acts. New York: Academic Press.Google Scholar
  15. Jackendoff, R. (1972).Semantic interpretation in generative grammar. Cambridge, MA: MIT Press.Google Scholar
  16. Kanerva (1990). Focusing on phonological phrases in Chichewa. In S. Inkelas & D. Zec (Eds.),The phonology-syntax connection (pp. 145–161). Chicago: University of Chicago Press.Google Scholar
  17. Krifka, M. (1991). A compositional semantics for multiple focus constructions.Proceedings of Semantics and Linguistic Theory (SALT) 1 (Cornell University Working Papers 11.)Google Scholar
  18. MacDonald, M., Pearlmutter, N., & Seidenberg, M. (1994). Lexical nature of syntatic ambiguity resolution.Psychological Review, 101, 676–703.CrossRefPubMedGoogle Scholar
  19. Marslen-Wilson, W., & Tyler, K. (1987). Against modularity. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp. 37–62). Cambridge, MA: MIT Press.Google Scholar
  20. Marslen-Wilson, W. D. (1987). Functional parallelism in spoken work recognition.Cognition, 25, 71–102.CrossRefPubMedGoogle Scholar
  21. Matin, E., Shao, K. C., & Boff, K. R. (1993). Saccadic overhead: Information processing time with and without saccades.Perception & Psychophysics, 53, 372–380.Google Scholar
  22. Mitchell, D. C., Corley, M. M. B., & Garnham, A. (1992). Effects of context in human sentence parsing: Evidence against a discourse-based proposal mechanism.Journal of Experimental Psychology: Learning, Memory and Cognition, 18, 69–88.Google Scholar
  23. Olson, D. (1970). Language and thought: Aspects of cognitive theory of semantics.Psychological Review, 77, 143–184.Google Scholar
  24. Perfetti, C. A. (1990). The cooperative language processors: Semantic influences in an autonomous syntax. In D. A. Balota, G. B. Flores d'Arcais, & K. Rayner (Eds.),Comprehension processes in reading. Hillsdale, NJ: Erlbaum.Google Scholar
  25. Pritchett, B. L. (1992).Grammatical competence and parsing performance. Chicago: University of Chicago Press.Google Scholar
  26. Rooth, M. (1985).Association with focus. Unpublished doctoral dissertation, University of Massachusetts, Amherst.Google Scholar
  27. Rooth, M. (1992). A theory of focus interpretation.Natural Language Semantics, 1, 75–116.Google Scholar
  28. Sedivy, J., Carlson, G., Tanenhaus, M., Spivey-Knowlton, M., & Eberhard, K. (1994). The cognitive function of contrast sets in processing focus constructions. InWorking Papers of the IBM Institute for Logic and Linguistics.Google Scholar
  29. Sedivy, J., Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Carlson, G. (1995). Using intonationally-marked presuppositional information in on-line language processing: Evidence from eye movements to a visual model.Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 375–380). Mahwah, NJ: LEA Associates.Google Scholar
  30. Spivey-Knowlton, M., & Sedivy, J. (1995). Parsing attachment ambiguities with multiple constraints.Cognition, 55, 227–267.CrossRefPubMedGoogle Scholar
  31. Spivey-Knowlton, M., Sedivy, J., Eberhard, K., & Tanenhaus, M. (1994). Psycholinguistic study of the interaction between language and vision. InProceedings of the 12th National Conference on Artificial Intelligence: Workshop on the Integration of Natural Language and Vision Processing.Google Scholar
  32. Spivey-Knowlton, M., & Tanenhaus, M. K. (1994). Referential context and syntactic ambiguity resolution. In C. Clifton, L. Frazier, & K. Rayner (Eds.),Perspectives on sentence processing. Hillsdale, NJ: Erlbaum.Google Scholar
  33. Spivey-Knowlton, M., & Tanenhaus, M. (1995).Syntactic ambiguity resolution in discourse: Modeling the effects of referential context and lexical frequency within an integration-competition framework. Manuscript submitted for publication.Google Scholar
  34. Spivey-Knowlton, M., Tanenhaus, M., Eberhard, K., & Sedivy, J. (1995). Eye-movements accompanying language and action in a visual context: Evidence against modularity. (Eds.),Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 25–30). Mahwah, NJ: LEA Associates.Google Scholar
  35. Spivey-Knowlton, M., Trueswell, J., & Tanenhaus, M. (1993). Context effects in syntactic ambiguity resolution: Discourse and semantic influences in parsing reduced relative clauses.Canadian Journal of Experimental Psychology, 37, 276–309.Google Scholar
  36. Steedman, M. (1987). Combinatory grammars and human language processing. In J. L. Garfield (Ed.),Modularity in knowledge representation and natural language understanding (pp. 187–205). Cambridge, MA: MIT Press.Google Scholar
  37. Swinney, D., & Osterhout, L. (1990). Interference generation during auditory language comprehension.The Psychology of Learning and Motivation, 25, 17–33.Google Scholar
  38. Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (1995). The interaction of visual and linguistic information in spoken language comprehension.Science, 268, 1632–1634.PubMedGoogle Scholar
  39. Tanenhaus, M., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (in press). Using eye-movements to study spoken language comprehension: Evidence for visually-mediated incremental interpretation. In T. Inui & J. McClelland (Eds.),Attention & Performance XVI: Integration in Perception and Communication.Google Scholar
  40. Tanenhaus, M., & Trueswell, J. (1995). Sentence comprehension. In J. Miller & P. Eimas (Eds.),Handbook of cognition and perception: Vol. 11. Speech and Language. San Diego, CA: Academic Press.Google Scholar
  41. Taraban, R., & McClelland, J. (1988). Constituent attachment and thematic role expectations.Journal of Memory and Language, 27, 597–632.CrossRefGoogle Scholar
  42. Taraban, R., & McClelland, J. (1990). Sentence comprehension: A multiple constraints view. In D. Balota, K. Rayner, & G. Flores d'Arcais (Eds.),Comprehension processes in reading. Hillsdale, NJ: Erlbaum.Google Scholar

Copyright information

© Plenum Publishing Corporation 1995

Authors and Affiliations

  • Kathleen M. Eberhard
    • 1
  • Michael J. Spivey-Knowlton
    • 1
  • Julie C. Sedivy
    • 1
  • Michael K. Tanenhaus
    • 1
  1. 1.Department of Brain and Cognitive SciencesUniversity of RochesterRochester

Personalised recommendations