Advertisement

HELPR: A Framework to Break the Barrier Across Domains in Spoken Dialog Systems

  • Ming Sun
  • Yun-Nung Chen
  • Alexander I. Rudnicky
Chapter
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 427)

Abstract

People usually interact with intelligent agents (IAs) when they have certain goals to be accomplished. Sometimes these goals are complex and may require interacting with multiple applications, which may focus on different domains. Current IAs may be of limited use in such cases and the user needs to directly manage the task at hand. An ideal personal agent would be able to learn, over time, these tasks spanning different resources. In this article, we address the problem of cross-domain task assistance in the context of spoken dialog systems, and describe our approach about discovering such tasks and how IAs learn to talk to users about the task being carried out. Specifically we investigate how to learn user activity patterns in a smartphone environment that span multiple apps and how to incorporate users’ descriptions about their high-level intents into human-agent interaction.

Keywords

Complex user intention Multi-domain Spoken dialog systems 

Notes

Acknowledgements

This work was supported in part by Yahoo! InMind, and by the General Motors Advanced Technical Center.

References

  1. 1.
    Harrison, C., Xiao, R., Schwarz, J., Hudson, S.E.: Touchtools: leveraging familiarity and skill with physical tools to augment touch interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2913–2916 (2014)Google Scholar
  2. 2.
    Sun, M., Chen, Y.N., Rudnicky, A.I.: Understanding user’s cross-domain intentions in spoken dialog systems. In: Proceedings of NIPS workshop on Machine Learning for SLU and Interaction (2015)Google Scholar
  3. 3.
    Chen, Y.N., Sun, M., Rudnicky, A.I.: Leveraging behavioral patterns of mobile applications for personalized spoken language understanding. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI), pp. 83–86 (2015)Google Scholar
  4. 4.
    Young, S.: Using POMDPs for dialog management. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 8–13 (2006)Google Scholar
  5. 5.
    Sun, M., Rudnicky, A.I., Winter, U.: User adaptation in automotive environments. In: Proceedings of the 4th International Conference on Advances in Human-Factors and Ergonomics (AHFE), pp. 156–165 (2012)Google Scholar
  6. 6.
    Pappu, A., Sun, M., Sridharan, S., Rudnicky, A.I.: Situated multiparty interactions between humans and agents. In: Proceedings of the 15th International Conference on Human-Computer Interaction: Interaction Modalities and Techniques, pp. 107–116 (2013)Google Scholar
  7. 7.
    Lunati, J.M., Rudnicky, A.I.: Spoken language interfaces: the OM system. In: Proceedings of the Conference on Human Factors in Computing Systems, pp. 453–454 (1991)Google Scholar
  8. 8.
    Rudnicky, A.I., Lunati, J.M., Franz, A.M.: Spoken language recognition in an office management domain. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 829–832. IEEE (1991)Google Scholar
  9. 9.
    Lin, B.S., Wang, H.M., Lee, L.S.: A distributed architecture for cooperative spoken dialogue agents with coherent dialogue state and history. In: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU), vol. 99, p. 4 (1999)Google Scholar
  10. 10.
    Nakano, M., Sato, S., Komatani, K., Matsuyama, K., Funakoshi, K., Okuno, H.G.: A two-stage domain selection framework for extensible multi-domain spoken dialogue systems. In: Proceedings of the 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 18–29. ACL (2011)Google Scholar
  11. 11.
    Chen, Y.N., Rudnicky, A.I.: Dynamically supporting unexplored domains in conversational interactions by enriching semantics with neural word embeddings. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 590–595. IEEE (2014)Google Scholar
  12. 12.
    Li, Q., Tür, G., Hakkani-Tür, D., Li, X., Paek, T., Gunawardana, A., Quirk, C.: Distributed open-domain conversational understanding framework with domain independent extractors. In: Proceedings of the IEEE Spoken Language Technology Workshop (SLT), pp. 566–571. IEEE (2014)Google Scholar
  13. 13.
    Ryu, S., Lee, D., Lee, I., Han, S., Lee, G.G., Kim, M., Kim, K.: A hierarchical domain model-based multi-domain selection framework for multi-domain dialog systems. In: Proceedings of the 24th International Conference on Computational Linguistics (ACL) (2012)Google Scholar
  14. 14.
    Ryu, S., Song, J., Koo, S., Kwon, S., Lee, G.G.: Detecting multiple domains from users utterance in spoken dialog system. In: Proceedings of the 6th International Workshop on Spoken Dialogue Systems (IWSDS), pp. 101–111 (2015)Google Scholar
  15. 15.
    Sun, M., Chen, Y.N., Hua, Z., Tamres-Rudnicky, Y., Dash, A., Rudnicky, A.I.: AppDialogue: Multi-app dialogues for intelligent assistants. In: Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC) (2016)Google Scholar
  16. 16.
    Lucchese, C., Orlando, S., Perego, R., Silvestri, F., Tolomei., G.: Identifying task-based sessions in search engine query logs. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining, pp. 277–286. ACM (2011)Google Scholar
  17. 17.
    Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., Tsujii, J.: BRAT: a web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 102–107. Association for Computational Linguistics (2012)Google Scholar
  18. 18.
    Bohus, D., Rudnicky, A.I.: Sorry, I didn’t catch that!-an investigation of non-understanding errors and recovery strategies. In: Proceedings of the 6th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) (2005)Google Scholar
  19. 19.
    Shen, X., Tan, B., Zhai., C.: Implicit user modeling for personalized search. In: Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pp. 824–831 (2005)Google Scholar
  20. 20.
    Sun, M., Chen, Y.N., Rudnicky., A.I.: Learning OOV through semantic relatedness in spoken dialog systems. In: Proceedings of the 16th Annual Conference of the International Speech Communication Association (Interspeech), pp. 1453–1457 (2015)Google Scholar
  21. 21.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the Workshop at International Conference on Learning Representations (ICLR) (2013)Google Scholar
  22. 22.
    Tibshirani, R., Walther, G., Hastie., T.: Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc.: Series B (Stat. Method.) 63, 411–423 (2001)Google Scholar
  23. 23.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley (2012)Google Scholar
  24. 24.
    Fiscus, J.G.: A post-processing system to yield reduced word error rates: recognizer output voting error reduction (ROVER). In: Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 347–352 (1997)Google Scholar
  25. 25.
    Ganesan, K., Zhai, C., Han, J.: Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pp. 340–348. ACL (2010)Google Scholar
  26. 26.
    Liu, F., Flanigan, J., Thomson, S., Sadeh, N., Smith, N.A.: Toward abstractive summarization using semantic representations. In: Proceedings of the 14th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie (NAACL-HLT), pp. 1077–1086 (2015)Google Scholar
  27. 27.
    Witten, I.H., Paynter, G.W., Frank, E., Gutwin, C., Nevill-Manning, C.G.: KEA: practical automatic keyphrase extraction. In: Proceedings of the 4th ACM Conference on Digital Libraries, pp. 254–255 (1999)Google Scholar
  28. 28.
    Medelyan, O.: Human-competitive automatic topic indexing. Ph.D. thesis, University of Waikato (2009)Google Scholar
  29. 29.
    Berry, M.W., Kogan., J.: Text Mining: Applications and Theory. Wiley (2010)Google Scholar
  30. 30.
    Mihalcea, R., Tarau, P.: TextRank: bringing order into texts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 404–411. ACL (2004)Google Scholar
  31. 31.
    Hulth, A.: Improved automatic keyword extraction given more linguistic knowledge. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 216–223. ACL (2003)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  • Ming Sun
    • 1
  • Yun-Nung Chen
    • 1
  • Alexander I. Rudnicky
    • 1
  1. 1.School of Computer ScienceCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations