Skip to main content

How Do Iconic Gestures Convey Visuo-Spatial Information? Bringing Together Empirical, Theoretical, and Simulation Studies

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7206)

Abstract

We investigate the question of how co-speech iconic gestures are used to convey visuo-spatial information in an interdisciplinary way, starting with a corpus-based empirical and theoretical perspective on how a typology of gesture form and a partial ontology of gesture meaning are related. Results provide the basis for a computational modeling approach that allows us to simulate the production of speaker-specific gesture forms to be realized with virtual agents. An evaluation of our simulation results and our methodology shows that the model is able to successfully approximate human gestural behavior use of iconic gestures, and moreover, that gestural behavior can improve how humans rate a virtual agent in terms of eloquence, competence, human-likeness, or likeability.

Keywords

  • Noun Phrase
  • Virtual Agent
  • Iconic Gesture
  • Gesture Data
  • Computational Modeling Approach

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   49.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language 58, 495–520 (2008)

    CrossRef  Google Scholar 

  2. Beattie, G., Shovelton, H.: An experimental investigation of the role of different types of iconic gesture in communication: A semantic feature approach. Gesture 1, 129–149 (2001)

    CrossRef  Google Scholar 

  3. Bergmann, K., Kopp, S.: GNetIc – Using Bayesian Decision Networks for Iconic Gesture Generation. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 76–89. Springer, Heidelberg (2009)

    CrossRef  Google Scholar 

  4. Bergmann, K., Kopp, S., Eyssel, F.: Individualized gesturing outperforms average gesturing–evaluating gesture production in virtual humans. In: Proceedings of IVA 2010, pp. 104–117. Springer, Heidelberg (2010)

    Google Scholar 

  5. Bergmann, K., Rieser, H., Kopp, S.: Regulating dialogue with gestures—towards an empirically grounded simulation with virtual agents. In: Proceedings of SigDial 2011. ACL, Portland (2011)

    Google Scholar 

  6. Hahn, F., Rieser, H.: Explaining speech gesture alignment in mm dialogue using gesture typology. In: Lupowski, P., Purver, M. (eds.) Proceedings of SemDial, pp. 99–111. Polish Society for Cognitive Science (2010)

    Google Scholar 

  7. Holler, J., Beattie, G.: How iconic gestures and speech interact in the representation of meaning: Are both aspects really integral to the process? Semiotica 146/1, 81–116 (2003)

    Google Scholar 

  8. Kendon, A.: Gesture—Visible Action as Utterance. Cambridge Univ. Press (2004)

    Google Scholar 

  9. Lücking, A., Bergmann, K., Hahn, F., Kopp, S., Rieser, H.: The Bielefeld speech and gesture alignment corpus (SaGA). In: Proceedings of the LREC 2010 Workshop on Multimodal Corpora (2010)

    Google Scholar 

  10. Rieser, H.: On Factoring Out a Gesture Typology from the Bielefeld Speech-and-Gesture-Alignment Corpus (SAGA). In: Kopp, S., Wachsmuth, I. (eds.) GW 2009. LNCS (LNAI), vol. 5934, pp. 47–60. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  11. Rieser, H.: How to disagree on a church-window’s shape using gesture. In: Hölker, K., Marello, C. (eds.) Dimensionen der Analyse von Texten und Diskursen, pp. 231–247. LIT Verlag, Münster (2011)

    Google Scholar 

  12. Sowa, T., Wachsmuth, I.: A model for the representation and processing of shape in coverbal iconic gestures. In: Proceedings of KogWis 2005, pp. 183–188 (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rieser, H., Bergmann, K., Kopp, S. (2012). How Do Iconic Gestures Convey Visuo-Spatial Information? Bringing Together Empirical, Theoretical, and Simulation Studies. In: Efthimiou, E., Kouroupetroglou, G., Fotinea, SE. (eds) Gesture and Sign Language in Human-Computer Interaction and Embodied Communication. GW 2011. Lecture Notes in Computer Science(), vol 7206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34182-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34182-3_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34181-6

  • Online ISBN: 978-3-642-34182-3

  • eBook Packages: Computer ScienceComputer Science (R0)