Skip to main content
SpringerLink
Log in

Real-Time Language Processing as Embodied and Embedded in Joint Action

  • Chapter
  • First Online:
Attention and Vision in Language Processing

Abstract

As the field of psycholinguistics gradually confronts the evidence that language is enmeshed with perceptual-motor processes, it may be in for a shock when it learns that perceptual-motor processes are themselves comprised of the relationship between organism and environment (e.g., ecological perception and active externalism). Therefore, not only is language interactive with perception and action, it is part of the continuous perception-action loop that a person develops with his/her environment. Crucially, one’s environment often has other people whose linguistic and perceptual-motor processes are also being externalised via their perception-action loops. As a result, real-time language processing appears to play a key role in allowing multiple people’s perception-action loops to become entangled and to produce interesting varieties of behavioural synchrony.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5(4), 307–331.

    Article  Google Scholar 

  • Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38(4), 419–439.

    Article  Google Scholar 

  • Anderson, M. L., Brumbaugh, J., & Şuben, A. (2010). Investigating functional cooperation in the human brain using simple graph-theoretic methods. In Computational neuroscience (pp. 31–42). New York: Springer.

    Google Scholar 

  • Arrow, H. (1997). Stability, bistability, and instability in small group influence patterns. Journal of Personality and Social Psychology, 72(1), 75.

    Article  Google Scholar 

  • Austin, J. L. (1962). How to do things with words. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Ballard, D., Hayhoe, M., & Pelz, J. (1995). Memory representations in natural tasks. Journal of Cognitive Neuroscience, 7(1), 66–80.

    Google Scholar 

  • Barsalou, L. W. (1999). Perceptions of perceptual symbols. Behavioral and Brain Sciences, 22(04), 637–660.

    Article  Google Scholar 

  • Bavelas, J., Kenwood, C., Johnson, T., & Phillips, B. (2002). An experimental study of when and how speakers use gestures to communicate. Gesture, 2(1), 1–17.

    Article  Google Scholar 

  • Bergmann, T., Dale, R., & Lupyan, G. (2013). The impact of communicative constraints on the emergence of a graphical communication system. In Proceedings of the Thirty-Fifth Annual Conference of the Cognitive Science Society (pp. 1887–1892).

    Google Scholar 

  • Cappella, J. N., & Planalp, S. (1981). Talk and silence sequences in informal conversations III: Interspeaker influence. Human Communication Research, 7(2), 117–132.

    Article  Google Scholar 

  • Chambers, C. G., Tanenhaus, M. K., & Magnuson, J. S. (2004). Actions and affordances in syntactic ambiguity resolution. Journal of Experimental Psychology. Learning, Memory, and Cognition, 30(3), 687.

    Article  Google Scholar 

  • Chang, C. H., Wade, M. G., & Stoffregen, T. A. (2009). Perceiving affordances for aperture passage in an environment–person–person system. Journal of Motor Behavior, 41(6), 495–500.

    Article  Google Scholar 

  • Chiu, E., & Spivey, M. J. (2012). The role of preview and incremental delivery on visual search. In Proceedings of the 34th Annual Conference of the Cognitive Science Society (pp. 216–221).

    Google Scholar 

  • Christiansen, M. H., & Kirby, S. (Eds.). (2003). Language evolution. New York: Oxford University Press.

    Google Scholar 

  • Clark, A. (2010). Supersizing the mind: Embodiment, action, and cognitive extension. Oxford University Press.

    Google Scholar 

  • Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press.

    Google Scholar 

  • Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. Perspectives on Socially Shared Cognition, 13, 127–149.

    Google Scholar 

  • Clark, H. H., & Schaefer, E. F. (1989). Contributing to discourse. Cognitive Science, 13(2), 259–294.

    Article  Google Scholar 

  • Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F., & Deltenre, P. (2002). Mismatch negativity evoked by the McGurk–MacDonald effect: A phonetic representation within short-term memory. Clinical Neurophysiology, 113(4), 495–506.

    Google Scholar 

  • Corbetta, M. (1998). Frontoparietal cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? Proceedings of the National Academy of Sciences, 95(3), 831–838.

    Article  Google Scholar 

  • Dahan, D., & Tanenhaus, M. K. (2005). Looking at the rope when looking for the snake: Conceptually mediated eye movements during spoken-word recognition. Psychonomic Bulletin & Review, 12(3), 453–459.

    Article  Google Scholar 

  • de la Rocha, O. (1985). The reorganization of arithmetic practice in the kitchen. Anthropology & Education Quarterly, 16(3), 193–198.

    Article  Google Scholar 

  • Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36(12), 1827–1837.

    Google Scholar 

  • Druskat, V. U., & Wolff, S. B. (2001). Building the emotional intelligence of groups. Harvard Business Review, 79(3), 80–91.

    Google Scholar 

  • Duncker, K., & Lees, L. S. (1945). On problem-solving. Psychological Monographs, 58(5), i (Whole No. 270).

    Google Scholar 

  • Eberhard, K. M., Spivey-Knowlton, M. J., Sedivy, J. C., & Tanenhaus, M. K. (1995). Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research, 24(6), 409–436.

    Article  Google Scholar 

  • Eguíluz, V. M., Zimmermann, M. G., Cela-Conde, C. J., & San Miguel, M. (2005). Cooperation and the emergence of role differentiation in the dynamics of social networks1. American Journal of Sociology, 110(4), 977–1008.

    Article  Google Scholar 

  • Farmer, T. A., Anderson, S. E., & Spivey, M. J. (2007). Gradiency and visual context in syntactic garden-paths. Journal of Memory and Language, 57(4), 570–595.

    Article  Google Scholar 

  • Fodor, J. A. (1983). The modularity of mind: An essay on faculty psychology. Cambridge: MIT press.

    Google Scholar 

  • Frazier, L. (1995). Constraint satisfaction as a theory of sentence processing. Journal of Psycholinguistic Research, 24(6), 437–468.

    Article  Google Scholar 

  • Freeman, J. B., & Ambady, N. (2011). A dynamic interactive theory of person construal. Psychological Review, 118(2), 247.

    Article  Google Scholar 

  • Fuchs, A., & Jirsa, V. K. (2008). Coordination: Neural, behavioral and social dynamics (Vol. 1). Berlin: Springer.

    Book  Google Scholar 

  • Fusaroli, R., Bahrami, B., Olsen, K., Roepstorff, A., Rees, G., Frith, C., & Tylén, K. (2012). Coming to terms quantifying the benefits of linguistic coordination. Psychological Science, 23, 931–939.

    Google Scholar 

  • Fusaroli, P., Vallar, R., Togliani, T., Khodadadian, E., & Caletti, G. (2002). Scientific publications in endoscopic ultrasonography: A 20-year global survey of the literature. Endoscopy, 34(06), 451–456.

    Article  Google Scholar 

  • Fusaroli, R., Gangopadhyay, N., & Tylén, K. (2013). The dialogically extended mind: Language as skilful intersubjective engagement. Cognitive Systems Research, 29, 31–39.

    Google Scholar 

  • Galantucci, B. (2005). An experimental study of the emergence of human communication systems. Cognitive Science, 29(5), 737–767.

    Article  Google Scholar 

  • Garrod, S., & Pickering, M. J. (2004). Why is conversation so easy? Trends in Cognitive Sciences, 8(1), 8–11.

    Article  Google Scholar 

  • Gibson, J. J. (1977). The concept of affordances. In Perceiving, acting, and knowing (pp. 67–82).

    Google Scholar 

  • Gibson, J. J. (1979). The ecological approach to visual perception. London: Psychology Press.

    Google Scholar 

  • Giles, H. (1973). Accent mobility: A model and some data. Anthropological Linguistics, 15, 87–105.

    Google Scholar 

  • Glenberg, A. M., & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9(3), 558–565.

    Article  Google Scholar 

  • Gold, J. I., & Shadlen, M. N. (2000). Representation of a perceptual decision in developing oculomotor commands. Nature, 404(6776), 390–394.

    Article  Google Scholar 

  • Goldin-Meadow, S. (1999). The role of gesture in communication and thinking. Trends in cognitive sciences, 3(11), 419–429.

    Article  Google Scholar 

  • Goldstone, R. L., & Gureckis, T. M. (2009). Collective behavior. Topics in Cognitive Science, 1(3), 412–438.

    Article  Google Scholar 

  • Grant, E. R., & Spivey, M. J. (2003). Eye movements and problem solving guiding attention guides thought. Psychological Science, 14(5), 462–466.

    Article  Google Scholar 

  • Huette, S., Winter, B., Matlock, T., & Spivey, M. (2012). Processing motion implied in language: eye-movement differences during aspect comprehension. Cognitive Processing, 13(1), 193–197.

    Article  Google Scholar 

  • Huettig, F., Quinlan, P. T., McDonald, S. A., & Altmann, G. (2006). Models of high-dimensional semantic space predict language-mediated eye movements in the visual world. Acta Psychologica, 121(1), 65–80.

    Article  Google Scholar 

  • Hutchins, E. (1995). Cognition in the Wild (Vol. 262082314). Cambridge, MA: MIT press.

    Google Scholar 

  • Isenhower, R. W., Richardson, M. J., Carello, C., Baron, R. M., & Marsh, K. L. (2010). Affording cooperation: Embodied constraints, dynamics, and action-scaled invariance in joint lifting. Psychonomic Bulletin & Review, 17(3), 342–347.

    Article  Google Scholar 

  • Jordan, J. S., & Hunsinger, M. (2008). Learned patterns of action-effect anticipation contribute to the spatial displacement of continuously moving stimuli. Journal of Experimental Psychology: Human Perception and Performance, 34(1), 113.

    Google Scholar 

  • Kaschak, M. P., & Borreggine, K. L. (2008). Temporal dynamics of the action–sentence compatibility effect. The Quarterly Journal of Experimental Psychology, 61(6), 883–895.

    Article  Google Scholar 

  • Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.

    Article  Google Scholar 

  • Kirsh, D. (1995). The intelligent use of space. Artificial Intelligence, 73(1), 31–68.

    Google Scholar 

  • Kirsh, D., & Maglio, P. (1994). On distinguishing epistemic from pragmatic action. Cognitive Science, 18(4), 513–549.

    Google Scholar 

  • Kita, S. (2000). How representational gestures help speaking. In Language and gesture (pp. 162–185).

    Google Scholar 

  • Knoblich, G., & Jordan, J. S. (2003). Action coordination in groups and individuals: learning anticipatory control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(5), 1006–1016.

    Google Scholar 

  • Kuhlen, A. K., Allefeld, C., & Haynes, J. D. (2012). Content-specific coordination of listeners’ to speakers’ EEG during communication. Frontiers in Human Neuroscience, 6, 266.

    Google Scholar 

  • Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.

    Google Scholar 

  • Marsh, K. L., Johnston, L., Richardson, M. J., & Schmidt, R. C. (2009). Toward a radically embodied, embedded social psychology. European Journal of Social Psychology, 39(7), 1217–1225.

    Article  Google Scholar 

  • Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word-recognition. Cognition, 25(1), 71–102.

    Article  Google Scholar 

  • Mataric, M. J. (1993, August). Designing emergent behaviors: From local interactions to collective intelligence. In Proceedings of the Second International Conference on Simulation of Adaptive Behavior (pp. 432–441).

    Google Scholar 

  • McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.

    Google Scholar 

  • McMurray, B., Tanenhaus, M. K., Aslin, R. N., & Spivey, M. J. (2003). Probabilistic constraint satisfaction at the lexical/phonetic interface: Evidence for gradient effects of within-category VOT on lexical access. Journal of Psycholinguistic Research, 32(1), 77–97.

    Article  Google Scholar 

  • McNeill, D. (2008). Gesture and thought. Chicago: University of Chicago Press.

    Google Scholar 

  • Meteyard, L., Zokaei, N., Bahrami, B., & Vigliocco, G. (2008). Visual motion interferes with lexical decision on motion words. Current Biology, 18(17), R732–R733.

    Article  Google Scholar 

  • Natale, M. (1975). Convergence of mean vocal intensity in dyadic communication as a function of social desirability. Journal of Personality and Social Psychology, 32(5), 790.

    Article  Google Scholar 

  • Nazir, T. A., Boulenger, V., Roy, A., Silber, B., Jeannerod, M., & Paulignan, Y. (2008). Language-induced motor perturbations during the execution of a reaching movement. The Quarterly Journal of Experimental Psychology, 61(6), 933–943.

    Article  Google Scholar 

  • Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. WH Freeman/Times Books/Henry Holt & Co.

    Google Scholar 

  • Proffitt, D. R. (2006). Distance perception. Current Directions in Psychological Science, 15(3), 131–135.

    Article  Google Scholar 

  • Pulvermüller, F., Hauk, O., Nikulin, V. V., & Ilmoniemi, R. J. (2005). Functional links between motor and language systems. European Journal of Neuroscience, 21(3), 793–797.

    Article  Google Scholar 

  • Read, S. J., & Miller, L. C. (2002). Virtual personalities: A neural network model of personality. Personality and Social Psychology Review, 6(4), 357–369.

    Article  Google Scholar 

  • Reali, F., Spivey, M. J., Tyler, M. J., & Terranova, J. (2006). Inefficient conjunction search made efficient by concurrent spoken delivery of target identity. Perception and Psychophysics, 68(6), 959–974.

    Article  Google Scholar 

  • Richardson, D. C., & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29(6), 1045–1060.

    Article  Google Scholar 

  • Richardson, D. C., Spivey, M. J., Barsalou, L. W., & McRae, K. (2003). Spatial representations activated during real-time comprehension of verbs. Cognitive Science, 27(5), 767–780.

    Article  Google Scholar 

  • Richardson, D. C., Dale, R., & Kirkham, N. Z. (2007a). The art of conversation is coordination common ground and the coupling of eye movements during dialogue. Psychological Science, 18(5), 407–413.

    Article  Google Scholar 

  • Richardson, D. C., Dale, R., & Spivey, M. J. (2007b). Eye movements in language and cognition. In Empirical methods in cognitive linguistics (pp. 323–344).

    Google Scholar 

  • Richardson, M. J., Marsh, K. L., & Baron, R. M. (2007c). Judging and actualizing intrapersonal and interpersonal affordances. Journal of Experimental Psychology: Human Perception and Performance, 33(4), 845.

    Google Scholar 

  • Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C. (2011). Interpersonal synergies. Frontiers in Psychology, 2, 38.

    Google Scholar 

  • Robbins, P., & Aydede, M. (2009). A short primer on situated cognition. In The Cambridge handbook of situated cognition (pp. 3–10).

    Google Scholar 

  • Roberts, M. E., & Goldstone, R. L. (2009). Adaptive group coordination. In Proceedings of the Thirty-First Annual Conference of the Cognitive Science Society (pp. 2698–2704).

    Google Scholar 

  • Schippers, M. B., Roebroeck, A., Renken, R., Nanetti, L., & Keysers, C. (2010). Mapping the information flow from one brain to another during gestural communication. Proceedings of the National Academy of Sciences, 107(20), 9388–9393.

    Article  Google Scholar 

  • Searle, J. R. (1969). Speech acts: An essay in the philosophy of language (Vol. 626). Cambridge: Cambridge University Press.

    Google Scholar 

  • Sebanz, N., Bekkering, H., & Knoblich, G. (2006). Joint action: bodies and minds moving together. Trends in Cognitive Sciences, 10(2), 70–76.

    Article  Google Scholar 

  • Senghas, A., & Coppola, M. (2001). Children creating language: How Nicaraguan sign language acquired a spatial grammar. Psychological Science, 12(4), 323–328.

    Article  Google Scholar 

  • Shannon, C. E. (1948). Key papers in the development of information theory. Bell Systems Technical Journal, 27, 623–656.

    Article  Google Scholar 

  • Shockley, K., Baker, A. A., Richardson, M. J., & Fowler, C. A. (2007). Articulatory constraints on interpersonal postural coordination. Journal of Experimental Psychology: Human Perception and Performance, 33(1), 201.

    Google Scholar 

  • Shockley, K., Richardson, D. C., & Dale, R. (2009). Conversation and coordinative structures. Topics in Cognitive Science, 1(2), 305–319.

    Article  Google Scholar 

  • Smith, M.A. (1992). Voices from the WELL: The logic of the virtual commons. Master’s thesis, UCLA.

    Google Scholar 

  • Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65(4), 235–241.

    Article  Google Scholar 

  • Spivey, M. J., Tyler, M. J., Eberhard, K. M., & Tanenhaus, M. K. (2001). Linguistically mediated visual search. Psychological Science, 12(4), 282–286.

    Article  Google Scholar 

  • Spivey, M. J., Tanenhaus, M. K., Eberhard, K. M., & Sedivy, J. C. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45(4), 447–481.

    Article  Google Scholar 

  • Stanfield, R. A., & Zwaan, R. A. (2001). The effect of implied orientation derived from verbal context on picture recognition. Psychological Science, 12(2), 153–156.

    Google Scholar 

  • Street, R. L. (1984). Speech convergence and speech evaluation in fact-finding interviews. Human Communication Research, 11(2), 139–169.

    Article  Google Scholar 

  • Tabor, W. (1995). Lexical change as nonlinear interpolation. In Proceedings of the 17th Annual Cognitive Science Conference (pp. 242–247).

    Google Scholar 

  • Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634.

    Article  Google Scholar 

  • Theiner, G., Allen, C., & Goldstone, R. L. (2010). Recognizing group cognition. Cognitive Systems Research, 11(4), 378–395.

    Article  Google Scholar 

  • Tollefsen, D. P. (2006). From extended mind to collective mind. Cognitive Systems Research, 7(2), 140–150.

    Article  Google Scholar 

  • Turvey, M. T. (1992). Affordances and prospective control: An outline of the ontology. Ecological Psychology, 4(3), 173–187.

    Article  Google Scholar 

  • Tylén, K., Weed, E., Wallentin, M., Roepstorff, A., & Frith, C. D. (2010). Language as a tool for interacting minds. Mind and Language, 25(1), 3–29.

    Article  Google Scholar 

  • Uttal, W. R. (2001). The new phrenology: The limits of localizing cognitive processes in the brain. Cambridge: The MIT Press.

    Google Scholar 

  • Williams, W. M., & Sternberg, R. J. (1988). Group intelligence: why some groups are better than others. Intelligence, 12(4), 351–377.

    Article  Google Scholar 

  • Yee, E., & Sedivy, J. C. (2006). Eye movements to pictures reveal transient semantic activation during spoken word recognition. Journal of Experimental Psychology. Learning, Memory, and Cognition, 32(1), 1.

    Article  Google Scholar 

  • Zwaan, R. A., Madden, C. J., Yaxley, R. H., & Aveyard, M. E. (2004). Moving words: Dynamic representations in language comprehension. Cognitive Science, 28(4), 611–619.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive and Information Sciences, University of California, Merced, CA, 95343, USA

    Lillian Rigoli & Michael J. Spivey

Authors
  1. Lillian Rigoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J. Spivey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael J. Spivey .

Editor information

Editors and Affiliations

  1. Center for Neural and Cognitive Sciences, University of Hyderabad, Hyderabad, Andhra Pradesh, India

    Ramesh Kumar Mishra

  2. Centre of Behavioural and Cognitive Sciences, University of Allahabad, Allahabad, Uttar Pradesh, India

    Narayanan Srinivasan

  3. Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

    Falk Huettig

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer India

About this chapter

Cite this chapter

Rigoli, L., Spivey, M.J. (2015). Real-Time Language Processing as Embodied and Embedded in Joint Action. In: Mishra, R., Srinivasan, N., Huettig, F. (eds) Attention and Vision in Language Processing. Springer, New Delhi. https://doi.org/10.1007/978-81-322-2443-3_1

Download citation

Publish with us

Policies and ethics

Search

Navigation