Skip to main content
SpringerLink
Log in

Preliminary Prosodic and Gestural Characteristics of Instructing Acts in Polish Task-Oriented Dialogues

  • Conference paper
Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5641))

  • 1584 Accesses

Abstract

In the present study, selected properties of multimodal instructing acts are discussed. Realisations of the instructing acts extracted from a corpus of task-oriented dialogues are analysed in terms of their syntactic structure, prosodic properties and accompanying gestures. The syntactic structures found in the material are similar to those found in earlier studies on map task dialogues. Deictic vocabulary is more frequent in gesture-supported instructions. The mean relative pitch range is similar to the values obtained for instructions in earlier studies and different from the values for syntactically similar questions. As opposite to verbally ill-formed instructions, the well-formed ones tend to contain at least one gestural stroke. It is shown that the relative range of pitch frequency is higher in the gesture-accompanied instructing acts. It is also noticed that prosody and gesture may play similar roles in utterances.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Jarmolowicz, E., Karpinski, M., Malisz, Z., Szczyszek, M.: Gesture, Prosody and Lexicon in Task-Oriented Dialogues: Multimedia Corpus Recording and Labelling. In: Esposito, A., Faundez-Zanuy, M., Keller, E., Marinaro, M. (eds.) COST Action 2102. LNCS, vol. 4775, pp. 99–110. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  2. Wahlster, W.: Dialogue Systems Go Multimodal: The SmartKom Experience. In: Wahlster, W. (ed.) SmartKom: Foundations of Multimodal Dialogue Systems. Cognitive Technologies Series, pp. 3–27. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  3. Karpinski, M.: From Speech and Gestures to Dialogue Acts. In: Esposito, A., Hussain, A., Marinaro, M., Martone, R. (eds.) Multimodal Signals 2008. LNCS, vol. 5398, pp. 164–169. Springer, Heidelberg (2009)

    Google Scholar 

  4. Kamp, H., Reyle, U.: From Discourse to Logic. In: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Springer, Heidelberg (1993)

    Google Scholar 

  5. Popescu-Belis, A.: Abstracting a Dialog Act Tagset for Meeting Processing. In: Proceedings of LREC 2004, Lisbon, Portugal, vol. IV, pp. 1415–1418 (2005)

    Google Scholar 

  6. Bunt, H.C., Romary, L.: Standardization in Multimodal Content Representation: Some Methodological Issues. In: Proceedings of LREC 2004, Lisbon, pp. 2219–2222 (2004)

    Google Scholar 

  7. Bunt, H., Girard, Y.: Designing an Open, Multidimensional Dialogue Act Taxonomy. In: Proceedings of DIALOR 2005 Workshop, Nancy (2005)

    Google Scholar 

  8. Traum, D.R.: 20 Questions for Dialogue Act Taxonomies. Journal of Semantics 17(1), 7–30 (2000)

    Article  Google Scholar 

  9. Kreutel, J., Matheson, C.: Obligations, Intentions, and the Notion of Conversational Games. In: Proc. Gotalog, 4th Workshop on the Semantics and Pragmatics of Dialogue (2000)

    Google Scholar 

  10. Bavelas, J.B., Chovil, N.: Visible acts of meaning. An Integrated Message Model of Language in Face-to-Face Dialogue. Journal of Language and Social Psychology 19(2), 163–194 (2000)

    Article  Google Scholar 

  11. Bavelas, J.B., Chovil, N., Coates, L., Roe, L.: Gestures specialized for dialogue. Personality and Social Psychology Bulletin 21, 394–405 (1995)

    Article  Google Scholar 

  12. Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language (2007)

    Google Scholar 

  13. Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: Paggio, P., Jokinen, K., Jönsson, A. (eds.) First Nordic Symposium on Multimodal Communication, pp. 7–22 (2003)

    Google Scholar 

  14. House, D.: Final rises and Swedish question intonation. In: Proceedings of Fonetik 2004, pp. 56–59 (2004)

    Google Scholar 

  15. House, D.: Perception of question intonation and facial gesture. In: Proceedings of Fonetik 2002, pp. 41–44 (2002)

    Google Scholar 

  16. Mast, M., Kompe, R., Harbeck, S., Kiessling, A., Niemann, H., Nöth, E., Schukat-Talamazzini, E.G., Warnke, V.: Dialog act classification with the help of prosody. In: Proceedings of ICSLP 1996, pp. 1732–1735 (1996)

    Google Scholar 

  17. Shriberg, E., Bates, E., Stolcke, A., Taylor, P., Jurafsky, D., Ries, K., Coccaro, N., Martin, R., Meteer, M., Van Ess-Dykema, C.: Can Prosody Aid the Automatic Classification of Dialogue Acts in Conversational Speech. Language and Speech 41(34), 439–487 (1998)

    Google Scholar 

  18. Wright, H., Poesio, M., Isard, S.: Using high level dialogue information for dialogue act recognition using prosodic features. In: Proceedings of DIAPRO 1999, pp. 139–143 (1999)

    Google Scholar 

  19. Fernandez, R., Picard, R.W.: Dialog act classification from prosodic features using support vector machines. In: Proceedings of Speech Prosody 2002, pp. 291–294 (2002)

    Google Scholar 

  20. Tamarit, V., Martinez-Hinarejos, C.-D.: Dialogue act labelling in the DIHANA corpus using prosody information. Jornadas en Tecnologia del Habla V, 183–186 (2008)

    Google Scholar 

  21. Venkataraman, A., Ferrer, L., Stolcke, A., Shriberg, E.: Training a prosody-based dialog act tagger from unlabeled data. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. I-272–I-275 (2003)

    Google Scholar 

  22. Rangarajan, V., Bangalore, S., Narayanan, S.S.: Exploiting prosodic features for dialog act tagging in a discriminative modeling framework. In: Proceedings of INTERSPEECH 2007, pp. 150–153 (2007)

    Google Scholar 

  23. Coria, S., Pineda, L.: Predicting Dialogue Acts from Prosodic Information. In: Gelbukh, A. (ed.) CICLing 2006. LNCS, vol. 3878, pp. 355–365. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Ang, J., Liu, Y., Shriberg, E.: Automatic dialog act segmentation and classification in multiparty meetings. In: Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1061–1064 (2005)

    Google Scholar 

  25. Karpinski, M.: Struktura i intonacja polskiego dialogu zadaniowego (The Structure and Intonation of Polish Task-Oriented Dialogues). Wydawnictwo Naukowe UAM, Poznan (2006)

    Google Scholar 

  26. Grabe, E., Karpinski, M.: Universal and Language-specific Aspects of Intonation in English and Polish. In: Grabe, E., Wright, D.G.S. (eds.) Oxford University Working Papers in Linguistics, Philology & Phonetics, vol. 8 (2003)

    Google Scholar 

  27. Karpinski, M., Szalkowska, E.: On intonation of question-type dialogue moves in Korean and Polish task-oriented dialogues. In: Proceedings of Speech Signal Annotation, Processing and Synthesis, Poznan (2006)

    Google Scholar 

  28. Karpinski, M.: The Intonational Realization of Requests in Polish Task-Oriented Dialogues. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 556–563. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  29. Fox, A.: Prosodic features and Prosodic Structures: The Phonology of Suprasegmentals. OUP, Oxford (2000)

    Google Scholar 

  30. McNeill, D.: Hand and Mind. In: What Gesture Reveals about Thought? University of Chicago Press, Chicago (1992)

    Google Scholar 

  31. Boersma, P., Wenink, D.: Doing Phonetics by Computer (A computer program; version 5.1) (2008)

    Google Scholar 

  32. Slama-Casacu, T.: Nonverbal components in message sequence: “Mixed syntax”. In: McCormack, W.C., Wurm, S.A. (eds.) Language and man: Anthropological issues, pp. 217–227. Mouton, The Hague (1976)

    Google Scholar 

  33. Bavelas, J.B.: Gestures as Part of Speech: Methodological Implications. Research on Language and Social Interaction 27(3), 201–221 (1994)

    Article  Google Scholar 

  34. McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2007)

    Google Scholar 

  35. Kendon, M.: Gesture: Visible Action as Utterance. CUP, Cambridge (2004)

    Book  Google Scholar 

  36. Bergmann, K., Kopp, S.: Co-expressivity of speech and gesture: Lessons for models of aligned speech and gesture production. In: Olivier, P., Kray, C. (eds.) AISB Annual Convention: Language, Speech and Gesture for Expressive Characters, pp. 153–158 (2007)

    Google Scholar 

  37. Buisine, S., Martin, J.-C.: The effects of speech–gesture cooperation in animated agents’ behavior in multimedia presentations. Interacting with Computers 19, 484–493 (2007)

    Article  Google Scholar 

  38. De Ruiter, J.P.: Some multimodal signals in humans. In: Van Der Sluis, I., Theune, M., Reiter, E., Krahmer, E. (eds.) Proceedings of the Workshop on Multimodal Output Generation MOG 2007, pp. 141–148 (2007)

    Google Scholar 

  39. Chovil, N.: Discourse-oriented facial displays in conversation. Research on Language and Social Interaction 25, 163–194 (1991/1992)

    Article  Google Scholar 

  40. Bavelas, J.B., Kenwood, C., Johnson, T., Phillips, B.: An experimental study of when and how speakers use gestures to communicate. Gesture 2(1), 1–18 (2002)

    Article  Google Scholar 

  41. Kendon, A.: Gesticulation and speech: two aspects of the process of utterance. In: Key, M.R. (ed.) The Relationship of Verbal and Nonverbal Communication, pp. 207–227. Mouton, The Hague (1980)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Linguistics and Center for Speech and Language Processing, Adam Mickiewicz University, Al. Niepodleglosci 4, 61-874, Poznan, Poland

    Maciej Karpiński

Authors
  1. Maciej Karpiński
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Psychology, Second University of Naples, and IIASS, Via G. Pellegrino 19, 84019, Vietri sul Mare, (SA), Italy

    Anna Esposito

  2. Institute of Photonics and Electronics, Academy of Sciences of the Czech Republic, Chaberská 57, 182 52, Prague 8, Czech Republic

    Robert Vích

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Karpiński, M. (2009). Preliminary Prosodic and Gestural Characteristics of Instructing Acts in Polish Task-Oriented Dialogues. In: Esposito, A., Vích, R. (eds) Cross-Modal Analysis of Speech, Gestures, Gaze and Facial Expressions. Lecture Notes in Computer Science(), vol 5641. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03320-9_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03320-9_22

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03319-3

  • Online ISBN: 978-3-642-03320-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation