Skip to main content
SpringerLink
Log in

User-Centred Spoken Dialogue Management

  • Chapter
Next Generation Intelligent Environments

  • 510 Accesses

Abstract

Adaptivity of intelligent environments to their surroundings provided by the ATRACO Spoken Dialogue Manager is only one means of adaptation. Recent work in Spoken Dialogue Systems focuses on the integration of user-centred adaptation means to alter the content, flow and structure of the ongoing dialogue. In this chapter, we introduce a general user-centred adaptation cycle, accompanied by two implemented adaptation approaches focusing respectively on short-term and long-term goals in human–computer interaction. After motivating the need for short-term and long-term goals to entail different adaptation mechanisms, we provide exemplary adaptation entities for each case with corresponding experiments and implementations. The short-term goal user satisfaction allows for detecting whether the user is not satisfied with the interaction and for triggering counter measures to improve the interaction. As a long-term goal, maintaining human–computer trust attempts to keep users still willing to use the system even if the interaction was confusing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The Lets Go domain will be introduced in more detail in Sect. 7.4.3.2.

  2. 2.

    The minimum number of exchanges to successfully complete the dialogue is 5.

  3. 3.

    All results for DCR and TSR are significantly different (chi-squared test). Significant differences in ADL (unpaired t-test) and AIQ (Mann–Whitney U test) with the respective strategy to the right are on the level of α < 0. 01 for system initiative (ADL) and mixed initiative (ADL) and on the level of α < 0. 05 for adaptive (AIQ), random (ADL, AIQ), and mixed initiative (AIQ). All other comparisons between non-neighbours are significant with α < 0. 01.

References

  1. Antoniou, G., van Harmelen, F.: Web ontology language: owl. In: Staab, S. (ed.) Handbook on Ontologies in Information Systems, pp. 76–92. Springer, Berlin (2003)

    Google Scholar 

  2. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011). Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

    Google Scholar 

  3. Engelbrecht, K.P., Gödde, F., Hartard, F., Ketabdar, H., Möller, S.: Modeling user satisfaction with hidden markov model. In: SIGDIAL ’09: Proceedings of the SIGDIAL 2009 Conference, pp. 170–177. Association for Computational Linguistics, Morristown, NJ, USA (2009)

    Google Scholar 

  4. Fraser, N.M.: The sundial speech understanding and dialogue project: results and implications for translation. In: Aslib Proceedings, vol. 46, pp. 141–148. MCB UP Ltd (1994)

    Google Scholar 

  5. Glass, A., McGuinness, D.L., Wolverton, M.: Toward establishing trust in adaptive agents. In: IUI ’08: Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 227–236. ACM, NY, USA (2008)

    Google Scholar 

  6. Gnjatović, M., Rösner, D.: Adaptive dialogue management in the nimitek prototype system. In: PIT ’08: Proceedings of the 4th IEEE Tutorial and Research Workshop on Perception and Interactive Technologies for Speech-Based Systems, pp. 14–25. Springer, Berlin (2008). doi:10.1007/978-3-540-69369-7_3

    Google Scholar 

  7. Hara, S., Kitaoka, N., Takeda, K.: Estimation method of user satisfaction using n-gram-based dialog history model for spoken dialog system. In: Calzolari, N., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., Rosner, M., Tapias, D. (eds.) Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10). European Language Resources Association (ELRA), Valletta, Malta (2010)

    Google Scholar 

  8. Heinroth, T., Denich, D.: Spoken interaction within the computed world: evaluation of a multitasking adaption spoken dialogue system. In: 35th Annual IEEE Computer Software and Applications Conference (COMPSAC), Munich, Germany, pp. 134–143. IEEE (2011)

    Google Scholar 

  9. Higashinaka, R., Minami, Y., Dohsaka, K., Meguro, T.: Issues in predicting user satisfaction transitions in dialogues: Individual differences, evaluation criteria, and prediction models. In: Lee, G., Mariani, J., Minker, W., Nakamura, S. (eds.) Spoken Dialogue Systems for Ambient Environments. Lecture Notes in Computer Science, vol. 6392, pp. 48–60. Springer, Berlin (2010). doi:10.1007/978-3-642-16202-2_5

    Chapter  Google Scholar 

  10. Higashinaka, R., Minami, Y., Dohsaka, K., Meguro, T.: Modeling user satisfaction transitions in dialogues from overall ratings. In: Proceedings of the SIGDIAL 2010 Conference, pp. 18–27. Association for Computational Linguistics, Tokyo, Japan (2010)

    Google Scholar 

  11. Hone, K.S., Graham, R.: Towards a tool for the subjective assessment of speech system interfaces (sassi). Nat. Lang. Eng. 6(3–4), 287–303 (2000). doi:10.1017/s1351324900002497

    Article  Google Scholar 

  12. Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1–2), 99–134 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  13. Larsson, S., Traum, D.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Nat. Lang. Eng. 6, 323–340 (2000)

    Article  Google Scholar 

  14. Lee, S., Eskenazi, M.: An unsupervised approach to user simulation: toward self-improving dialog systems. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 50–59. Association for Computational Linguistics (2012)

    Google Scholar 

  15. Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors J. Hum. Factors Ergon. Soc. 46(1), 50–80 (2004)

    Article  Google Scholar 

  16. Lim, B.Y., Dey, A.K., Avrahami, D.: Why and why not explanations improve the intelligibility of context-aware intelligent systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’09, pp. 2119–2128. ACM, NY (2009)

    Google Scholar 

  17. Lindgaard, G., Dudek, C.: What is this evasive beast we call user satisfaction? Interact. Comput. 15(3), 429–452 (2003)

    Article  Google Scholar 

  18. Litman, D., Pan, S.: Designing and evaluating an adaptive spoken dialogue system. User Model. User-Adap. Inter. 12(2–3), 111–137 (2002). doi:10.1023/a:1015036910358

    Article  MATH  Google Scholar 

  19. Madsen, M., Gregor, S.: Measuring human-computer trust. In: Proceedings of the 11th Australasian Conference on Information Systems, pp. 6–8 (2000)

    Google Scholar 

  20. Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18(1), 50–60 (1947)

    Article  MATH  MathSciNet  Google Scholar 

  21. Mayer, R.C., Davis, J.H., Schoorman, F.D.: An integrative model of organizational trust. Acad. Manag. Rev. 20(3), 709–734 (1995)

    Google Scholar 

  22. McTear, M.F.: Spoken Dialogue Technology – Toward the Conversational User Interface. Springer, London (2004)

    Book  Google Scholar 

  23. Muir, B.M.: Trust in automation: part i. theoretical issues in the study of trust and human intervention in automated systems. In: Ergonomics, pp. 1905–1922 (1992)

    Google Scholar 

  24. Müller, F., Späth, C., Geier, T., Biundo, S.: Exploiting expert knowledge in factored POMDPs. In: Proceedings of the 20th European Conference on Artificial Intelligence (ECAI 2012), pp. 606–611 (2012)

    Google Scholar 

  25. Nothdurft, F., Honold, F., Kurzok, P.: Using explanations for runtime dialogue adaptation. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, pp. 63–64. ACM, New York (2012)

    Google Scholar 

  26. Oshry, M., Auburn, R., Baggia, P., Bodell, M., Burke, D., Burnett, D., Candell, E., Carter, J., Mcglashan, S., Lee, A., Porter, B., Rehor, K.: Voice extensible markup language (voicexml) version 2.1. Technical Report, W3C – Voice Browser Working Group (2007)

    Google Scholar 

  27. Pan, A.S., Litman, D.J., Pan, S.: Empirically evaluating an adaptable spoken dialogue system diane j. litman. In: Proceedings of the 7th International Conference on User Modeling, pp. 55–64 (1999)

    Google Scholar 

  28. Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. Factors J. Hum. Factors Ergon. Soc. 39(2), 230–253 (1997)

    Article  Google Scholar 

  29. Potel, M.: MVP: model-view-presenter the taligent programming model for C++ and java. Technical Report, Taligent Inc (1996). http://www.wildcrest.com/Potel/Portfolio/mvp.pdf

  30. Raux, A., Bohus, D., Langner, B., Black, A.W., Eskenazi, M.: Doing research on a deployed spoken dialogue system: one year of let’s go! experience. In: Proceedings of the International Conference on Speech and Language Processing (ICSLP) (2006)

    Google Scholar 

  31. Sanner, S.: Relational dynamic influence diagram language (rddl): language description (2010). Http://users.cecs.anu.edu.au/ ssanner/IPPC2011/RDDL.pdf

    Google Scholar 

  32. Schmitt, A., Schatz, B., Minker, W.: Modeling and predicting quality in spoken human-computer interaction. In: Proceedings of the SIGDIAL 2011 Conference, pp. 173–184. Association for Computational Linguistics, Portland, Oregon, USA (2011)

    Google Scholar 

  33. Schmitt, A., Schatz, B., Minker, W.: A statistical approach for estimating user satisfaction in spoken human-machine interaction. In: Proceedings of the IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), pp. 1–6. IEEE, Amman, Jordan (2011)

    Google Scholar 

  34. Schmitt, A., Ultes, S., Minker, W.: A parameterized and annotated spoken dialog corpus of the cmu let’s go bus information system. In: International Conference on Language Resources and Evaluation (LREC), pp. 3369–337 (2012)

    Google Scholar 

  35. Sidorov, M., Ultes, S., Schmitt, A.: Emotions are a personal thing: Towards speaker-adaptive emotion recognition. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 4836–4840 (2014)

    Google Scholar 

  36. Silver, D., Veness, J.: Monte-carlo planning in large POMDPs. In: NIPS, pp. 2164–2172 (2010)

    Google Scholar 

  37. Spearman, C.E.: The proof and measurement of association between two things. Am. J. Psychol. 15, 88–103 (1904)

    Google Scholar 

  38. Ultes, S., Minker, W.: Improving interaction quality recognition using error correction. In: Proceedings of the 14th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 122–126. Association for Computational Linguistics (2013). http://www.aclweb.org/anthology/W/W13/W13-4018

  39. Ultes, S., Minker, W.: Interaction quality estimation in spoken dialogue systems using hybrid-hmms. In: Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 208–217. Association for Computational Linguistics (2014). http://www.aclweb.org/anthology/W14-4328

  40. Ultes, S., Minker, W.: Managing adaptive spoken dialogue for intelligent environments. J. Ambient Intell. Smart Environ. 6(5), 523–539 (2014). doi:10.3233/ais-140275

    Google Scholar 

  41. Ultes, S., Heinroth, T., Schmitt, A., Minker, W.: A theoretical framework for a user-centered spoken dialog manager. In: Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop, pp. 241–246. Springer, New York (2011)

    Google Scholar 

  42. Ultes, S., Schmitt, A., Minker, W.: Attention, sobriety checkpoint! can humans determine by means of voice, if someone is drunk…and can automatic classifiers compete? In: Proceedings of the 12th Annual Conference of the International Speech Communication Association (INTERSPEECH 2011), pp. 3221–3224 (2011)

    Google Scholar 

  43. Ultes, S., ElChabb, R., Minker, W.: Application and evaluation of a conditioned hidden markov model for estimating interaction quality of spoken dialogue systems. In: Mariani, J., Devillers, L., Garnier-Rizet, M., Rosset, S. (eds.) Proceedings of the 4th International Workshop on Spoken Language Dialog System (IWSDS), pp. 141–150. Springer, New York (2012)

    Google Scholar 

  44. Ultes, S., Schmitt, A., Minker, W.: Towards quality-adaptive spoken dialogue management. In: NAACL-HLT Workshop on Future directions and needs in the Spoken Dialog Community: Tools and Data (SDCTD 2012), pp. 49–52. Association for Computational Linguistics, Montréal, Canada (2012). http://www.aclweb.org/anthology/W12-1819

  45. Ultes, S., ElChabb, R., Schmitt, A., Minker, W.: Jachmm: a java-based conditioned hidden markov model library. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013, Vancouver, BC, Canada, pp. 3213–3217. IEEE (2013)

    Google Scholar 

  46. Ultes, S., Schmitt, A., Minker, W.: On quality ratings for spoken dialogue systems – experts vs. users. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 569–578. Association for Computational Linguistics (2013)

    Google Scholar 

  47. Ultes, S., Dikme, H., Minker, W.: Dialogue management for user-centered adaptive dialogue. In: Proceedings of the 5th International Workshop On Spoken Dialogue Systems (IWSDS) (2014). http://www.uni-ulm.de/fileadmin/website_uni_ulm/allgemein/2014_iwsds/iwsds2014_lp_ultes.pdf

  48. Ultes, S., Dikme, H., Minker, W.: First insight into quality-adaptive dialogue. In: International Conference on Language Resources and Evaluation (LREC), pp. 246–251 (2014)

    Google Scholar 

  49. Ultes, S., Kraus, M., Schmitt, A., Minker, W.: On objective performance measures of quality-adaptive spoken dialogue—and their correlation with interaction quality. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2015, submitted)

    Google Scholar 

  50. Ultes, S., Platero Sánchez, M.J., Schmitt, A., Minker, W.: Analysis of an extended interaction quality corpus. In: Proceedings of the 6th International Workshop On Spoken Dialogue Systems (IWSDS) (2015). Accepted for publication

    Google Scholar 

  51. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    Book  MATH  Google Scholar 

  52. Williams, J.D.: At&t statistical dialog toolkit (2010). http://www2.research.att.com/sw/tools/asdt/

  53. Williams, J.D.: Incremental partition recombination for efficient tracking of multiple dialog states. In: Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, Dallas, Texas, pp. 5382–5385. IEEE (2010)

    Google Scholar 

  54. Williams, J., Young, S.: Partially observable markov decision processes for spoken dialog systems. Comput. Speech Lang. 21(2), 231–422 (2007)

    Article  Google Scholar 

  55. Young, S., Schatzmann, J., Weilhammer, K., Ye, H.: The hidden information state approach to dialog management. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2007 (ICASSP 2007), vol. 4 (2007)

    Google Scholar 

  56. Young, S.J., Gačić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The hidden information state model: a practical framework for POMDP-based spoken dialogue management. Comput. Speech Lang. 24(2), 150–174 (2010)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute Communications Engineering, Ulm University, Ulm, 89081, Germany

    Florian Nothdurft, Stefan Ultes & Wolfgang Minker

Authors
  1. Florian Nothdurft
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Ultes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfgang Minker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Nothdurft .

Editor information

Editors and Affiliations

  1. University of Ulm, Ulm, Germany

    Stefan Ultes

  2. University of Ulm, Ulm, Germany

    Florian Nothdurft

  3. University of Ulm, Ulm, Germany

    Tobias Heinroth

  4. University of Ulm, Ulm, Germany

    Wolfgang Minker

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Nothdurft, F., Ultes, S., Minker, W. (2016). User-Centred Spoken Dialogue Management. In: Ultes, S., Nothdurft, F., Heinroth, T., Minker, W. (eds) Next Generation Intelligent Environments. Springer, Cham. https://doi.org/10.1007/978-3-319-23452-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23452-6_7

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23451-9

  • Online ISBN: 978-3-319-23452-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation