Preliminary Prosodic and Gestural Characteristics of Instructing Acts in Polish Task-Oriented Dialogues

  • Maciej Karpiński
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5641)


In the present study, selected properties of multimodal instructing acts are discussed. Realisations of the instructing acts extracted from a corpus of task-oriented dialogues are analysed in terms of their syntactic structure, prosodic properties and accompanying gestures. The syntactic structures found in the material are similar to those found in earlier studies on map task dialogues. Deictic vocabulary is more frequent in gesture-supported instructions. The mean relative pitch range is similar to the values obtained for instructions in earlier studies and different from the values for syntactically similar questions. As opposite to verbally ill-formed instructions, the well-formed ones tend to contain at least one gestural stroke. It is shown that the relative range of pitch frequency is higher in the gesture-accompanied instructing acts. It is also noticed that prosody and gesture may play similar roles in utterances.


task-oriented dialogue instructing acts gesture intonation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Jarmolowicz, E., Karpinski, M., Malisz, Z., Szczyszek, M.: Gesture, Prosody and Lexicon in Task-Oriented Dialogues: Multimedia Corpus Recording and Labelling. In: Esposito, A., Faundez-Zanuy, M., Keller, E., Marinaro, M. (eds.) COST Action 2102. LNCS, vol. 4775, pp. 99–110. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Wahlster, W.: Dialogue Systems Go Multimodal: The SmartKom Experience. In: Wahlster, W. (ed.) SmartKom: Foundations of Multimodal Dialogue Systems. Cognitive Technologies Series, pp. 3–27. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Karpinski, M.: From Speech and Gestures to Dialogue Acts. In: Esposito, A., Hussain, A., Marinaro, M., Martone, R. (eds.) Multimodal Signals 2008. LNCS, vol. 5398, pp. 164–169. Springer, Heidelberg (2009)Google Scholar
  4. 4.
    Kamp, H., Reyle, U.: From Discourse to Logic. In: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Springer, Heidelberg (1993)Google Scholar
  5. 5.
    Popescu-Belis, A.: Abstracting a Dialog Act Tagset for Meeting Processing. In: Proceedings of LREC 2004, Lisbon, Portugal, vol. IV, pp. 1415–1418 (2005)Google Scholar
  6. 6.
    Bunt, H.C., Romary, L.: Standardization in Multimodal Content Representation: Some Methodological Issues. In: Proceedings of LREC 2004, Lisbon, pp. 2219–2222 (2004)Google Scholar
  7. 7.
    Bunt, H., Girard, Y.: Designing an Open, Multidimensional Dialogue Act Taxonomy. In: Proceedings of DIALOR 2005 Workshop, Nancy (2005)Google Scholar
  8. 8.
    Traum, D.R.: 20 Questions for Dialogue Act Taxonomies. Journal of Semantics 17(1), 7–30 (2000)CrossRefGoogle Scholar
  9. 9.
    Kreutel, J., Matheson, C.: Obligations, Intentions, and the Notion of Conversational Games. In: Proc. Gotalog, 4th Workshop on the Semantics and Pragmatics of Dialogue (2000)Google Scholar
  10. 10.
    Bavelas, J.B., Chovil, N.: Visible acts of meaning. An Integrated Message Model of Language in Face-to-Face Dialogue. Journal of Language and Social Psychology 19(2), 163–194 (2000)CrossRefGoogle Scholar
  11. 11.
    Bavelas, J.B., Chovil, N., Coates, L., Roe, L.: Gestures specialized for dialogue. Personality and Social Psychology Bulletin 21, 394–405 (1995)CrossRefGoogle Scholar
  12. 12.
    Bavelas, J., Gerwing, J., Sutton, C., Prevost, D.: Gesturing on the telephone: Independent effects of dialogue and visibility. Journal of Memory and Language (2007)Google Scholar
  13. 13.
    Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: Paggio, P., Jokinen, K., Jönsson, A. (eds.) First Nordic Symposium on Multimodal Communication, pp. 7–22 (2003)Google Scholar
  14. 14.
    House, D.: Final rises and Swedish question intonation. In: Proceedings of Fonetik 2004, pp. 56–59 (2004)Google Scholar
  15. 15.
    House, D.: Perception of question intonation and facial gesture. In: Proceedings of Fonetik 2002, pp. 41–44 (2002)Google Scholar
  16. 16.
    Mast, M., Kompe, R., Harbeck, S., Kiessling, A., Niemann, H., Nöth, E., Schukat-Talamazzini, E.G., Warnke, V.: Dialog act classification with the help of prosody. In: Proceedings of ICSLP 1996, pp. 1732–1735 (1996)Google Scholar
  17. 17.
    Shriberg, E., Bates, E., Stolcke, A., Taylor, P., Jurafsky, D., Ries, K., Coccaro, N., Martin, R., Meteer, M., Van Ess-Dykema, C.: Can Prosody Aid the Automatic Classification of Dialogue Acts in Conversational Speech. Language and Speech 41(34), 439–487 (1998)Google Scholar
  18. 18.
    Wright, H., Poesio, M., Isard, S.: Using high level dialogue information for dialogue act recognition using prosodic features. In: Proceedings of DIAPRO 1999, pp. 139–143 (1999)Google Scholar
  19. 19.
    Fernandez, R., Picard, R.W.: Dialog act classification from prosodic features using support vector machines. In: Proceedings of Speech Prosody 2002, pp. 291–294 (2002)Google Scholar
  20. 20.
    Tamarit, V., Martinez-Hinarejos, C.-D.: Dialogue act labelling in the DIHANA corpus using prosody information. Jornadas en Tecnologia del Habla V, 183–186 (2008)Google Scholar
  21. 21.
    Venkataraman, A., Ferrer, L., Stolcke, A., Shriberg, E.: Training a prosody-based dialog act tagger from unlabeled data. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. I-272–I-275 (2003)Google Scholar
  22. 22.
    Rangarajan, V., Bangalore, S., Narayanan, S.S.: Exploiting prosodic features for dialog act tagging in a discriminative modeling framework. In: Proceedings of INTERSPEECH 2007, pp. 150–153 (2007)Google Scholar
  23. 23.
    Coria, S., Pineda, L.: Predicting Dialogue Acts from Prosodic Information. In: Gelbukh, A. (ed.) CICLing 2006. LNCS, vol. 3878, pp. 355–365. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  24. 24.
    Ang, J., Liu, Y., Shriberg, E.: Automatic dialog act segmentation and classification in multiparty meetings. In: Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1061–1064 (2005)Google Scholar
  25. 25.
    Karpinski, M.: Struktura i intonacja polskiego dialogu zadaniowego (The Structure and Intonation of Polish Task-Oriented Dialogues). Wydawnictwo Naukowe UAM, Poznan (2006)Google Scholar
  26. 26.
    Grabe, E., Karpinski, M.: Universal and Language-specific Aspects of Intonation in English and Polish. In: Grabe, E., Wright, D.G.S. (eds.) Oxford University Working Papers in Linguistics, Philology & Phonetics, vol. 8 (2003)Google Scholar
  27. 27.
    Karpinski, M., Szalkowska, E.: On intonation of question-type dialogue moves in Korean and Polish task-oriented dialogues. In: Proceedings of Speech Signal Annotation, Processing and Synthesis, Poznan (2006)Google Scholar
  28. 28.
    Karpinski, M.: The Intonational Realization of Requests in Polish Task-Oriented Dialogues. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 556–563. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  29. 29.
    Fox, A.: Prosodic features and Prosodic Structures: The Phonology of Suprasegmentals. OUP, Oxford (2000)Google Scholar
  30. 30.
    McNeill, D.: Hand and Mind. In: What Gesture Reveals about Thought? University of Chicago Press, Chicago (1992)Google Scholar
  31. 31.
    Boersma, P., Wenink, D.: Doing Phonetics by Computer (A computer program; version 5.1) (2008)Google Scholar
  32. 32.
    Slama-Casacu, T.: Nonverbal components in message sequence: “Mixed syntax”. In: McCormack, W.C., Wurm, S.A. (eds.) Language and man: Anthropological issues, pp. 217–227. Mouton, The Hague (1976)Google Scholar
  33. 33.
    Bavelas, J.B.: Gestures as Part of Speech: Methodological Implications. Research on Language and Social Interaction 27(3), 201–221 (1994)CrossRefGoogle Scholar
  34. 34.
    McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2007)Google Scholar
  35. 35.
    Kendon, M.: Gesture: Visible Action as Utterance. CUP, Cambridge (2004)CrossRefGoogle Scholar
  36. 36.
    Bergmann, K., Kopp, S.: Co-expressivity of speech and gesture: Lessons for models of aligned speech and gesture production. In: Olivier, P., Kray, C. (eds.) AISB Annual Convention: Language, Speech and Gesture for Expressive Characters, pp. 153–158 (2007)Google Scholar
  37. 37.
    Buisine, S., Martin, J.-C.: The effects of speech–gesture cooperation in animated agents’ behavior in multimedia presentations. Interacting with Computers 19, 484–493 (2007)CrossRefGoogle Scholar
  38. 38.
    De Ruiter, J.P.: Some multimodal signals in humans. In: Van Der Sluis, I., Theune, M., Reiter, E., Krahmer, E. (eds.) Proceedings of the Workshop on Multimodal Output Generation MOG 2007, pp. 141–148 (2007)Google Scholar
  39. 39.
    Chovil, N.: Discourse-oriented facial displays in conversation. Research on Language and Social Interaction 25, 163–194 (1991/1992)CrossRefGoogle Scholar
  40. 40.
    Bavelas, J.B., Kenwood, C., Johnson, T., Phillips, B.: An experimental study of when and how speakers use gestures to communicate. Gesture 2(1), 1–18 (2002)CrossRefGoogle Scholar
  41. 41.
    Kendon, A.: Gesticulation and speech: two aspects of the process of utterance. In: Key, M.R. (ed.) The Relationship of Verbal and Nonverbal Communication, pp. 207–227. Mouton, The Hague (1980)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Maciej Karpiński
    • 1
  1. 1.Institute of Linguistics and Center for Speech and Language ProcessingAdam Mickiewicz UniversityPoznanPoland

Personalised recommendations