Skip to main content
SpringerLink
Log in

Communicative Gestures Facilitate Problem Solving for Both Communicators and Recipients

  • Chapter
Model-Based Reasoning in Science, Technology, and Medicine

Part of the book series: Studies in Computational Intelligence ((SCI,volume 64))

Summary. Gestures are a common, integral part of communication. Here, we investigate the roles of gesture and speech in explanations, both for communicators and recipients. Communicators explained how to assemble a simple object, using either speech with gestures or gestures alone. Gestures used for explaining included pointing and exhibiting to indicate parts, action models to demonstrate assembly, and gestures used to convey narrative structure. Communicators using gestures alone learned assembly better, making fewer assembly errors than those communicating via speech with gestures. Recipients understood and learned better from gesture-only instructions than from speech-only instructions. Gestures demonstrating action were particularly crucial, suggesting that superiority of gestures to speech may reside, at least in part, in compatibility between gesture and action.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alibali, M.M., Kita, S., Young, A.: Gesture and the process of speech production: We think, therefore we gesture. Language and Cognitive Processes 15 (2000) 593-613

    Article  Google Scholar 

  2. Alibali, M.W., DiRusso, A.A.: The function of gesture in learning to count: More than keeping track. Cognitive Development 14 (1999) 37-56

    Article  Google Scholar 

  3. Alibali, M.W., Bassok, M., Olseth Solomon, K., Syc, S.E., Goldin-Meadow, S.: Illuminating mental representation through speech and gesture. Psychological Science 10 (1999) 327-333

    Article  Google Scholar 

  4. Alibali, M.W., Heath, D., Myers, H.: Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language 44 (2001) 169-188

    Article  Google Scholar 

  5. Anderson, R.C., Ortony, A.: (1975). On putting apples into bottles: A problem of polysemy. Cognitive Psychology, 7, 167-180

    Article  Google Scholar 

  6. Barsalou, L.W.: Perceptual symbol systems. Behavioral and Brain Sciences 22 (1999) 577-609

    Google Scholar 

  7. Clark, H.H.: Using language. Cambridge: Cambridge University Press. (1996)

    Google Scholar 

  8. Clark, H.H., Krych, M.A.: Speaking while monitoring addressees for understand- ing. Journal of Memory and Language 50 (2004) 62-81

    Article  Google Scholar 

  9. Cohen, A.A.: The communicative function of hand gestures. Journal of Com- munication 27 (1977) 54-63

    Article  Google Scholar 

  10. Cohen, A.A., Harrison, R. P.: Intentionality in the use of hand illustrators in face-to-face communication situations. Journal of Personality and Social Psy-chology 28 (1973) 276-279

    Article  Google Scholar 

  11. Cohen, J.D., MacWhinney, B., Flatt, M., Provost, J.: PsyScope: An interactive graphic system for designing and controlling experiments in the psychology lab-oratory using Macintosh computers. Behavior Research, Methods, Instruments & Computers 25 (1993) 257-271

    Google Scholar 

  12. Dittmann, A.T., Llewelyn, L.G.: Body movement and speech rhythm in social conversation. Journal of Personality and Social Psychology 23 (1969) 283-292

    Google Scholar 

  13. Emmorey, K., Tversky, B., Taylor, H.A.: Using space to describe space: Perspec-tive in speech, sign, and gesture. Journal of Spatial Cognition and Computation 2 (2000) 157-180

    Article  Google Scholar 

  14. Engle, R.A.: Not channels but composite signals: Speech, gesture, diagrams, and object demonstrations are integrated in multimodal explanations. In M.A. Gernsbacher & S.J. Derry, eds., Spatial language: Cognition and computational perspectives. Kluwer Academic, Dordrecht, The Netherlands (1998)

    Google Scholar 

  15. Frick-Horbury, D., Guttentag, R.E.: The effects of restricting hand gesture pro-duction on lexical retrieval and free recall. American Journal of Psychology 111 (1998) 43-62

    Article  Google Scholar 

  16. Glenberg, A.M.: What memory is for. Behavioral and Brain Sciences 2 (1997) 1-55

    Google Scholar 

  17. Glenberg, A.M., Kaschak, M.P.: Grounding language in action. Psychonomic Bulletin & Review 9 (2002) 558-565

    Google Scholar 

  18. Graham, J.A., Argyle, M.: A cross-cultural study of the communication of extra-verbal meaning by gestures. International Journal of Psychology 10 (1975) 57-67

    Article  Google Scholar 

  19. Graham, J.A., Heywood, S.: The effects of elimination of hand gestures and of verbal codability on speech performance. European Journal of Social Psychology 5 (1975) 189-195

    Article  Google Scholar 

  20. Goldin-Meadow, S.: Hearing gesture: How our hands help us think. Belknap Press, Cambridge (2003)

    Google Scholar 

  21. Goldin-Meadow, S., Alibali, M.W., Church, R.B.: Transitions in concept acquisi-tion: Using the hand to read the mind. Psychological Review 100 (1993) 279-297

    Article  Google Scholar 

  22. Haviland, J.: Pointing, gesture spaces, and mental maps. In D. McNeill (Ed.), Language and gesture. Cambridge: Cambridge University Press (2000) 13-46

    Google Scholar 

  23. Heiser, J., Phan, D., Agrawala, M., Tversky, B., Hanrahan, P.: Identification and validation of cognitive design principles for automated generation of assembly instructions. Proceedings of Advanced Visual Interfaces (2004) 311-319

    Google Scholar 

  24. Heiser, J., Tversky, B.: Descriptions and depictions in acquiring complex sys-tems. In Proceedings of the 24th annual meeting of the Cognitive Science Society. Fairfax, VA. (2002)

    Google Scholar 

  25. Heiser, J., Tversky, B.: Cognitive design principles for visualizations: Revealing and instantiating. In Proceedings of the 25th Annual Meeting of the Cognitive Science Society. Boston, MA. (2003)

    Google Scholar 

  26. Hewes, G.: Primate communication and the gestural origins of language. Current Anthropology 14 (1973) 5-24

    Article  Google Scholar 

  27. Kendon, A.: Gesture: Visible action in utterance. New York: Cambridge University Press. (2004)

    Google Scholar 

  28. Kita, S., Ö zyürek, A.: What does cross-linguistic variation in semantic coordi-nation of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48 (2003) 16-32

    Article  Google Scholar 

  29. Krauss, R.M.: Why do we gesture when we speak? Current Directions in Psy-chological Science 7 (1998) 54-59

    Article  Google Scholar 

  30. Krauss, R.M., Hadar, U.: The role of speech-related arm/hand gestures in word retrieval. In L. Messing & R. Campbell (Eds.), Gesture, speech, and sign. New York, NY: Oxford University Press. (1999) 93-116

    Google Scholar 

  31. Krauss, R.M., Chen, Y., Chawla, P.: Nonverbal behavior and nonverbal commu-nication: What do conversational hand gestures tell us? In M.P. Zanna (Ed.). Advances in experimental social psychology 28. San Diego, CA: Academic Press. (1996)

    Google Scholar 

  32. Levinson, S.: Pragmatics. Cambridge: Cambridge University Press. (1983)

    Google Scholar 

  33. McNeill, D.: Hand and mind: What gestures reveal about thought. University of Chicago Press, Chicago (1992)

    Google Scholar 

  34. McNeill, D.: Analogic/analytic representations and cross-linguistic differences in thinking for speaking. Cognitive Linguistics Special Issue: Language Acquisition 11 (2000) 43-60

    Google Scholar 

  35. Özyürek, A.: So speakers design their cospeech gestures for their addressees. The effects of addressee location on representational gestures. Journal of Memory and Language 46 (2002) 688-704

    Article  Google Scholar 

  36. Quintilian: In H.E. Butler (Ed. and Trans.), Institutes of oratory. London: William Heinemann Press. (1920)

    Google Scholar 

  37. Rauscher, F.H., Krauss, R.M., Chen, Y.: Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science 7 (1996) 226-231

    Article  Google Scholar 

  38. Riseborough, M.G.: Physiographic gestures as decoding facilitators: Three experiments exploring a neglected facet of communication. Journal of Nonverbal Behavior 5 (1981) 172-183

    Article  Google Scholar 

  39. Rogers, W.T.: The contribution of kinesic illustrators toward the comprehension of verbal behavior within utterances. Human Communication Research 5 (1978) 54-62

    Article  Google Scholar 

  40. Rogers, W.T.: The relevance of body motion cues to both functional and dys-functional communicative behavior. Journal of Communication Disorders 12 (1979) 273-282

    Article  Google Scholar 

  41. Schooler, J.W., Engstler-Schooler, T.Y.: Verbal overshadowing of visual memo-ries: Some things are better left unsaid. Cognitive Psychology 22 (1990) 36-71

    Google Scholar 

  42. Tucker, M., Ellis, R.: On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance 24 (1998) 830-846

    Article  Google Scholar 

  43. Tversky, B.: Spatial schemas in depictions. In M. Gattis, ed., Spatial schemas and abstract thought. MIT Press, Cambridge (2001) 79-111

    Google Scholar 

  44. Vandenburg, S.G., Kuse, A.R.: Mental rotations, a group test of three-dimensional spatial visualization. Perceptual and Motor Skills 47 (1978) 599-604

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Stanford University, Jordan Hall, Building 420, 94305, Stanford, CA, USA

    Sandra C. Lozano

  2. Department of Psychology, Stanford University, Jordan Hall, Building 420, 94305, Stanford, CA, USA

    Barbara Tversky

Authors
  1. Sandra C. Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara Tversky
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Pavia, Pizza Botta 6, 27100, Pavia, Italy

    Lorenzo Magnani

  2. Sun Yat-sen University, 510275, Guangzhou, P.R.China

    Ping Li

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Lozano, S.C., Tversky, B. (2007). Communicative Gestures Facilitate Problem Solving for Both Communicators and Recipients. In: Magnani, L., Li, P. (eds) Model-Based Reasoning in Science, Technology, and Medicine. Studies in Computational Intelligence, vol 64. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71986-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-71986-1_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-71985-4

  • Online ISBN: 978-3-540-71986-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation