Abstract
Visibility of speakers’ conversational hand gestures has been shown to facilitate listeners’ comprehension. Is this true for all types of gestures? In Experiment 1, speakers were videotaped describing apartment layouts from memory. When describing complex compared to simple layouts, speakers used more representational gestures. In particular, they used more iconic–deictic gestures, or hand movements that represent both an object or action, and direction or location. This suggests that the frequency of complex representational gestures increases as a function of task difficulty. To assess the extent to which these gestures facilitate speech comprehension, in Experiment 2 a new group of participants either watched the videos of the apartment descriptions (audio + video condition) or only heard the descriptions (audio-only condition) and drew each corresponding layout. Although drawing accuracy did not differ as a function of condition, the more iconic–deictic gestures produced during the apartment descriptions, the less accurate listeners’ drawings were. Together, findings from Experiments 1 and 2 suggest that iconic–deictic gestures reflect task difficulty for the speaker and do not necessarily facilitate comprehension for the listener.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The study was designed to additionally explore differences in first and second language production (English and Spanish), though this is not the focus of this paper: For the purpose of this paper, we will only describe results of the apartments that were described in English. Order of language (first or second) was counterbalanced across participants. A native English speaker gave all instructions in English.
We are grateful for this suggestion by an anonymous reviewer of our manuscript.
The model summarized in Table 5 reveals that the overall use of speakers’ other gestures during a given description, after accounting for the use of iconic-deictic gestures, was positive, b = 0.31(0.12), F(1, 151) = 6.13, p < 0.02 (95 % CI 0.062; 0.55). This positive association in overall gesture rate is driven by the positive association between beat gestures and apartment drawing accuracy revealed in Table 6. As can be seen in Table 6, when accounting for beat gestures, the association between total gestures used and accuracy is not significant.
References
Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating and thinking about spatial information. Spatial Cognition and Computation, 5, 307–331. doi:10.1207/s15427633scc0504_2.
Alibali, M. W., Heath, D. C., & Myers, H. J. (2001). Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language, 44, 169–188. doi:10.1006/jmla.2000.2752.
Alibali, M. W., Kita, S., & Young, A. (2000). Gesture and the process of speech production: We think, therefore we gesture. Language and Cognitive Processes, 15, 593–613. doi:10.1080/016909600750040571.
Allen, L. Q. (2000). Nonverbal accommodations in foreign language teacher talk. Applied Language Learning, 11, 155–176.
Bavelas, J. B., Black, B., Chovil, N., Lemery, C. R., & Mullett, J. (1988). Form and function in motor mimicry. Topographic evidence that the primary function is communicative. Human Communication Research, 14, 275–299. doi:10.1111/j.1468-2958.1988.tb00158.x.
Bavelas, J. B., Gerwing, J., Sutton, C., & Prevost, D. (2008). Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language, 58, 495–520. doi:10.1016/j.jml.2007.02.004.
Beattie, G. W., & Shovelton, H. K. (1999). Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica, 123, 1–2. doi:10.1515/semi.1999.123.1-2.1.
Butterworth, B., & Hadar, U. (1989). Gesture, speech, and computational stages: A reply to McNeill. Psychological Review, 96, 168–174. doi:10.1037/0033-295X.96.1.168.
Casasanto, D. (2013). Gesture and language processing. In H. Pashler (Ed.), Encyclopedia of the mind (Vol. 7, pp. 373–375). Thousand Oaks, CA: Sage Publications.
Clark, H. H., & Krych, M. A. (2004). Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50, 62–81. doi:10.1016/j.jml.2003.08.004.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement. Educational and Psychological Measurement, 20, 37–46. doi:10.1177/001316446002000104.
De Ruiter, J. P. (1998). Gesture and speech production. Doctoral dissertation at Catholic University of Nijmegen, Netherlands.
Drolet, A. L., & Morris, M. W. (2000). Rapport in conflict resolution: Accounting for how face-to-face contact fosters mutual cooperation in mixed-motive conflicts. Journal of Experimental Social Psychology, 36, 26–50. doi:10.1006/jesp.1999.1395.
Duncan, S. (2002). Gesture, verb aspect, and the nature of iconic imagery in natural discourse. Gesture, 2, 183–206. doi:10.1075/gest.2.2.04dun.
Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: The Belknap Press.
Goldin-Meadow, S., & Alibali, M. W. (2013). Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 64, 257–283. doi:10.1146/annurev-psych-113011-143802.
Hadar, U., Dar, R., & Teitelman, A. (2001). Gesture during speech in first and second language: Implications for lexical retrieval. Gesture, 1, 151–165. doi:10.1075/gest.1.2.04had.
Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137, 297–315. doi:10.1037/a0022128.
Hostetter, A. B., & Alibali, M. W. (2007). Raise your hand if you’re spatial: Relations between verbal and spatial skills and gesture production. Gesture, 7, 73–95. doi:10.1075/gest.7.1.05hos.
Hostetter, A. B., Alibali, M. W., & Kita, S. (2007). I see it in my hand’s eye: Representational gestures are sensitive to conceptual demands. Language and Cognitive Processes, 22, 313–336. doi:10.1080/01690960600632812.
Iverson, J., Capirici, O., Longobardi, E., & Caselli, M. (1999). Gesturing in mother–child interactions. Cognitive Development, 14, 57–75. doi:10.1016/S0885-2014(99)80018-5.
Jacobs, N., & Garnham, A. (2007). The role of conversational hand gestures in a narrative task. Journal of Memory and Language, 56, 291–303. doi:10.1016/j.jml.2006.07.011.
Jesse, A., & Massaro, D. W. (2010). Seeing a singer helps comprehension of the song’s lyrics. Psychonomic Bulletin & Review, 17, 323–328. doi:10.3758/PBR.17.3.323.
Kendon, A. (1983). Gesture and speech: How they interact. In J. M. Weimann & R. P. Harrison (Eds.), Nonverbal interaction. Beverly Hills, CA: Sage Publications.
Kimbara, I. (2006). On gestural mimicry. Gesture, 6, 39–61. doi:10.1075/gest.6.1.03kim.
Kita, S. (2000). How representational gestures help speaking. In D. McNeill (Ed.), Language and gesture (pp. 162–185). Cambridge, UK: Cambridge University Press.
Krahmer, E., & Swerts, M. (2007). Effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception. Journal of Memory and Language, 57, 396–414. doi:10.1016/j.jml.2007.06.005.
Krauss, R. M. (1998). Why do we gesture when we speak? Current Directions in Psychological Science, 7, 54–59. doi:10.1111/1467-8721.ep13175642.
Krauss, R. M., & Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In L. S. Messing & R. Campbell (Eds.), Gesture, speech, and sign (pp. 93–116). Oxford, UK: Oxford University Press.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago, USA: University of Chicago Press.
Melinger, A., & Kita, S. (2007). Conceptualisation load triggers gesture production. Language and Cognitive Processes, 22, 473–500. doi:10.1080/01690960600696916.
Mol, L., Krahmer, E., Maes, A., & Swerts, M. (2011). Seeing and being seen: The effects on gesture production. Journal of Computer Mediated Communications, 17, 77–100. doi:10.1111/j.1083-6101.2011.01558.x.
Morsella, E., & Krauss, R. M. (2004). The role of gestures in spatial working memory and speech. American Journal of Psychology, 117, 411–424. doi:10.2307/4149008.
Pardo, J. S. (2006). On phonetic convergence during conversational interaction. Journal of the Acoustical Society of America, 119, 2382–2393. doi:10.1121/1.2178720.
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169–226. doi:10.1017/S0140525X04000056.
Pine, K. J., Bird, H., & Kirk, E. (2007). The effects of prohibiting gestures on children’s lexical retrieval ability. Developmental Science, 10, 747–754. doi:10.1111/j.1467-7687.2007.00610.x.
Purdy, J. M., Ney, P., & Balakrishnan, P. V. (2000). The impact of communication media on negotiation outcomes. International Journal of Conflict Management, 11, 162–187. doi:10.1108/eb022839.
Rauscher, F. B., Krauss, R. M., & Chen, Y. (1996). Gesture, speech and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226–231. doi:10.1111/j.1467-9280.1996.tb00364.x.
Ravizza, S. (2003). Movement and lexical access: Do noniconic gestures aid in retrieval? Psychonomic Bulletin & Review, 10, 610–615. doi:10.3758/BF03196522.
SAS Institute. (2001). SAS system for Microsoft Windows. Cary, NC: SAS Institute.
Wallbott, H. G. (1995). Congruence, contagion, and motor mimicry: Mutualities in nonverbal exchange. In J. Markova, C. F. Graumann, & K. Foppa (Eds.), Mutualities in dialogue (pp. 82–98). Cambridge, UK: Cambridge University Press.
Wesp, R., Hesse, J., Keutmann, D., & Wheaton, K. (2001). Gestures maintain spatial imagery. American Journal of Psychology, 114, 591–600. doi:10.2307/1423612.
Author information
Authors and Affiliations
Corresponding author
Appendix
Rights and permissions
About this article
Cite this article
Suppes, A., Tzeng, C.Y. & Galguera, L. Using and Seeing Co-speech Gesture in a Spatial Task. J Nonverbal Behav 39, 241–257 (2015). https://doi.org/10.1007/s10919-015-0207-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10919-015-0207-7