Dialogue Act Annotation with the ISO 24617-2 Standard

  • Harry Bunt
  • Volha Petukhova
  • David Traum
  • Jan Alexandersson
Chapter

Abstract

This chapter describes recent and ongoing annotation efforts using the ISO 24617-2 standard for dialogue act annotation. Experimental studies are reported on the annotation by human annotators and by annotation machines of some of the specific features of the ISO annotation scheme, such as its multidimensional annotation of communicative functions, the recognition of each of its nine dimensions, and the recognition of dialogue act qualifiers for certainty, conditionality, and sentiment. The construction of corpora of dialogues, annotated according to ISO 24617-2, is discussed, including the recent DBOX and DialogBank corpora.

References

  1. 1.
    Alexandersson, J., Buschbeck-Wolf, B., Fujinami, T., Kipp, M., Koch, S., Maier, E., et al. (1998). Dialogue acts in VERBMOBIL-2 (second edition). Verbmobil Report 226. Saarbrücken: DFKI.Google Scholar
  2. 2.
    Allen, J., & Core, M. (1997). DAMSL: Dialogue act markup in several layers (Draft 2.1). Technical Report. Rochester, NY: University of Rochester.Google Scholar
  3. 3.
    Allwood, J. (1992). On dialogue cohesion. Gothenburg University, Department of Linguistics.Google Scholar
  4. 4.
    Bunt, H. (1994). Context and dialogue control. Think Quarterly, 3(1), 19–31.Google Scholar
  5. 5.
    Bunt, H. (2000). Dialogue pragmatics and context specification. In H. Bunt & W. Black (Eds.), Abduction, belief and context in dialogue. Studies in computational pragmatics (pp. 81–150). Amsterdam: John Benjamins.Google Scholar
  6. 6.
    Bunt, H. (2006). Dimensions in dialogue annotation. In Proceedings 5th International Conference on Language Resources and Evaluation (LREC 2006), Genova, Paris. ELRA.Google Scholar
  7. 7.
    Bunt, H. (2009). The DIT\( {}^{++} \) taxonomy for functional for dialogue markup. In D. Heylen, C. Pelachaud, R. Catizone, & D. Traum (Eds.), Proceedings of EDAML-AAMAS Workshop “Towards a Standard Markup Language for Embodied Dialogue Acts”, Budapest (pp. 36–48).Google Scholar
  8. 8.
    Bunt, H. (2011). Multifunctionality in dialogue. Computer, Speech and Language, 25, 222–245.CrossRefGoogle Scholar
  9. 9.
    Bunt, H. (2015). On the principles of semantic annotation. In Proceedings 11th Joint ACL-ISO Workshop on Interoperable Semantic Annotation (ISA-11), London (pp. 1–13).Google Scholar
  10. 10.
    Bunt, H., Alexandersson, J., Carletta, J., Choe, J.-W., Fang, A., Hasida, K., et al. (2010). Towards and ISO standard for dialogue act annotation. In Proceedings 7th International Conference on Language Resources and Evaluation (LREC 2010), Malta, Paris. ELDA.Google Scholar
  11. 11.
    Bunt, H., Alexandersson, J., Choe, J.-W., Fang, A., Hasida, K., Petukhova, V., et al. (2012). ISO 24617-2: A semantically-based standard for dialogue annotation. In Proceedings of 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul. Paris: ELDA.Google Scholar
  12. 12.
    Bunt, H., Fang, A., Cao, J., Liu, X., & Petukhova, V. (2013). Issues in the addition of ISO standard annotations to the Switchboard corpus. In Proceedings 9th Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation (ISA-9), Potsdam (pp. 59–70).Google Scholar
  13. 13.
    Bunt, H., Kipp, M., & Petukhova, V. (2012). Using DiAML and ANVIL for multimodal dialogue annotation. In Proceedings 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul. Paris: ELRA.Google Scholar
  14. 14.
    Bunt, H., Petukhova, V., Malchanau, A., & Wijnhoven, K. (2016). The DialogBank. In Proceedings 10th International Conference on Language Resources and Evaluation (LREC 2016), Portoroz, Slovenia. Paris: ELRA.Google Scholar
  15. 15.
    Calhoun, S., Carletta, J., Brenier, J., Mayo, N., Jurafsky, D., Steedman, M., et al. (2010). The NXT-format Switchboard corpus: A rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language Resources and Evaluation, 44(4), 387–419.CrossRefGoogle Scholar
  16. 16.
    Carletta, J., Isard, S., Kowtko, J., & Doherty-Sneddon, G. (1996). HCRC dialogue structure coding manual. Technical Report HCRC/TR-82, University of Edinburgh.Google Scholar
  17. 17.
    Cohen, J. (1960). A coefficient of agreement for nominal scales. Education and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
  18. 18.
    Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–261.CrossRefGoogle Scholar
  19. 19.
    Dhillon, R., Bhagat, S., Carvey, H., & Schriberg, E. (2004). Meeting recorder project: Dialogue labelling guide. ICSI Technical Report TR-04-002. University of California at Berkeley.Google Scholar
  20. 20.
    Fang, A., Cao, J., Bunt, H., & Liu, X. (2011). Relating the semantics of dialogue acts to linguistic properties: A machine learning perspective through lexical cues. In Proceedings IEEE-ICSC 2011 Workshop on Semantic Annotation for Computational Linguistic Resources, Stanford, CA.Google Scholar
  21. 21.
    Fang, A., Cao, J., Bunt, H., & Liu, X. (2012). The annotation of the Switchboard corpus with the new ISO standard for dialogue act analysis. In Proceedings 8th Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation (ISA-8), Pisa (pp. 13–18).Google Scholar
  22. 22.
    Fang, A., Cao, J., Bunt, H., & Liu, X. (2012). Applicability verification of a new ISO standard for dialogue act annotation with the Switchboard corpus. In Proceedings of EACL 2012 Workshop on Innovative Hybrid Approaches to the Processing of Textual Data, Avignon.Google Scholar
  23. 23.
    Geertzen, J. (2007). DitAT: A flexible tool to support web-based dialogue annotation. In Proceedings 7th International Conference on Computational Semantics (IWCS-7), Tilburg (pp. 320–323).Google Scholar
  24. 24.
    Geertzen, J., & Bunt, H. (2006). Measuring annotator agreement in a complex, hierarchical dialogue act schema. In Proceedings SIGDIAL 2006, Sydney.Google Scholar
  25. 25.
    Geertzen, J., Girard, Y., & Morante, R. (2004). The DIAMOND project. In Proceedings of 8th Workshop on the Semantics and Pragmatics of Dialogue (CATALOG 2004), Barcelona.Google Scholar
  26. 26.
    Geertzen, J., Petukhova, V., & Bunt, H. (2008). Evaluating dialogue act tagging with naive and expert annotators. In Proceedings 6th International Conference on Language Resources and Evaluation (LREC 2008), Marrakech. Paris: ELDA.Google Scholar
  27. 27.
    Hovy, E., & Maier, E. (1995). Parsimonious or profligate: How many and which discourse structure relations? ISI Research Report. Marina del Rey: Information Sciences Institute, University of Southern California.Google Scholar
  28. 28.
    Ide, N., & Romary, L. (2004). International standard for a linguistic annotation framework. Natural Language Engineering, 10, 211–225.CrossRefGoogle Scholar
  29. 29.
    ISO (2011). ISO 24612: Language Resource Management - Linguistic Annotation Framework (LAF). Geneva: ISO.Google Scholar
  30. 30.
    ISO (2012). ISO 24617-2: Language Resource Management - Semantic Annotation Framework (SemAF) - Part 2: Dialogue Acts. Geneva: ISO.Google Scholar
  31. 31.
    ISO (2016). ISO 24617-6: Language Resource Management - Semantic Annotation Framework (SemAF) - Part 6: Principles of Semantic Annotation. Geneva: ISO.Google Scholar
  32. 32.
    Jurafsky, D., Shriberg, E., & Biasca, D. (1997). Switchboard SWBD-DAMSL shallow-discourse-function annotation: Coders manual, Draft 1.3. University of Colorado.Google Scholar
  33. 33.
    Lendvai, P., van den Bosch, A., Krahmer, E., & Canisius, S. (2004). Memory-based robust interpretation of recognised speech. In Proceedings 9th International Conference on Speech and Computer (SPECOM’04), St. Petersburg (pp. 415–422).Google Scholar
  34. 34.
    Lesch, S., Kleinbauer, T., & Alexandersson, J. (2005). A new metric for the evaluation of dialog act classification. In Proceedings 9th Workshop on the Semantics and Pragmatics of Dialogue (DIALOR), Nancy.Google Scholar
  35. 35.
    Meteer, M., & Taylor, A. (1995). Dysflency annotation stylebook for the Switchboard corpus. Washington: Linguistic Data Consortium.Google Scholar
  36. 36.
    Petukhova, V. (2011). Multidimensional dialogue modelling. Ph.D. dissertation. Tilburg University.Google Scholar
  37. 37.
    Petukhova, V., & Bunt, H. (2011). Incremental dialogue act understanding. In Proceedings Ninth International Conference on Computational Semantics (IWCS 2011), Oxford (pp. 235–244).Google Scholar
  38. 38.
    Petukhova, V., Gropp, M., Klakow, D., Eigner, G., Topf, M., Srb, S., et al. (2014). The DBOX corpus collection of spoken human-human and human-machine dialogues. In Proceedings 9th International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland.Google Scholar
  39. 39.
    Petukhova, V., Prévot, L., & Bunt, H. (2011). Multi-level discourse relations between dialogue units. In Proceedings 6th Joint ACL-ISO Workshop on Interoperable Semantic Annotation (ISA-6), Oxford (pp. 18–28).Google Scholar
  40. 40.
    Popescu-Belis, A. (2005). Dialogue acts: One or more dimensions? ISSCO Working Paper 62. Geneva: ISSCO.Google Scholar
  41. 41.
    Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., et al. (2008). The Penn Discourse TreeBank 2.0. In Proceedings 6th International Conference on Language Resources and Systems (LREC 2008), Marrakech.Google Scholar
  42. 42.
    Prüst, H., Minnen, G., & Beun, R.-J. (1984). Transcriptie dialooogesperiment juni/juli 1984. IPO Rapport 481. Institute for Perception Research, Eindhoven University of Technology.Google Scholar
  43. 43.
    Reithinger, N., & Klesen, M. (1997). Dialogue act classification using language models. In Proceedings of Eurospeech-97 (pp. 2235–2238).Google Scholar
  44. 44.
    Samuel, K., Carberry, S., & Vijay-Shanker, K. (1998). Dialogue act tagging with transformation-based learning. In Proceedings ACL 1998, Montreal (pp. 1150–1156).Google Scholar
  45. 45.
    Stolcke, A., Res, K., Coccaro, K., Shriberg, E., Bates, R., Jurafsky, D., et al. (2000). Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3), 339–373.CrossRefGoogle Scholar
  46. 46.
    Traum, D. (2000). 20 questions on dialogue act taxonomies. Journal of Semantics, 17(1), 7–30.CrossRefGoogle Scholar
  47. 47.
    Zimmermann, M., Lui, Y., Shriberg, E., & Stolcke, A. (2005). Toward joint segmentation and classification of dialogue acts in multiparty meetings. In Proceedings of the Multimodal Interaction and Related Machine Learning Algorithms Workshop (MLMI-05) (pp. 187–193). Berlin: Springer.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Harry Bunt
    • 1
  • Volha Petukhova
    • 2
  • David Traum
    • 3
  • Jan Alexandersson
    • 4
  1. 1.Tilburg Center for Cognition and Communication (TiCC)Tilburg UniversityTilburgThe Netherlands
  2. 2.Spoken Language Systems GroupSaarland UniversitySaarbrückenGermany
  3. 3.Institute fro Creative TechnologiesUniversity of Southern CaliforniaLos AngelesUSA
  4. 4.Department of Intelligent User Interfaces DFKIGernan Research Center for Artificial IntelligenceSaarbrückenGermany

Personalised recommendations