Abstract
Next-generation autonomous agents will be expected to converse with people to achieve their mutual goals. Human-machine dialogue, however, is challenged by noisy acoustic data, and by people’s preference for more natural interaction. This paper describes an ambitious project that embeds human subjects in a spoken dialogue system. It collects a rich and novel data set, including spoken dialogue, human behavior, and system features. During data collection, subjects were restricted to the same databases, action choices, and noisy automated speech recognition output as a spoken dialogue system. This paper mines that data to learn how people manage the problems that arise during dialogue under such restrictions. Two different approaches to successful, goal-directed dialogue are identified this way, from which supervised learning can predict appropriate dialogue choices. The resultant models can then be incorporated into an autonomous agent that seeks to assist its user.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Levin, E., Passonneau, R.: A WOz Variant with Contrastive Conditions. In: Interspeech Satelite Workshop, Dialogue on Dialogues: Multidisciplinary Evaluation of Speech-Based Interactive Systems (2006)
Passonneau, R.J., Epstein, S.L., Ligorio, T., Gordon, J., Bhutada, P.: Learning About Voice Search for Spoken Dialogue Systems. In: 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pp. 840–848 (2010)
Bohus, D., Rudnicky, A.: The Ravenclaw Dialogue Management Framework: Architecture and Systems. Computers in Speech and Language 23, 332–361 (2009)
Raux, A., Langner, B., Black, A., Eskenazi, M.: Let’s Go Public! Taking a Spoken Dialog System to the Real World. In: Interspeech 2005, Eurospeech (2005)
Seneff, S., Hurley, E., Lau, R., Pao, C., Schmid, P., Zue, V.: Galaxy II: A Reference Architecture for Conversational System Development. In: 5th International Conference on Spoken Language Systems, ICSLP 1998 (1998)
Raux, A., Eskenazi, M.: A Multi-Layer Architecture for Semi-Synchronous Event-Driven Dialogue Management. In: IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2007 (2007)
Raux, A., Eskenazi, M.: Optimizing Endpointing Thresholds Using Dialogue Features in a Spoken Dialogue System. In: SIGdial 2008 (2008)
Huggins-Daines, D., Kumar, M., Chan, A., Black, A.W., Ravishankar, M., Rudnicky, A.: Pocketsphinx: A Free, Real-Time Continuous Speech Recognition System for Hand-Held Devices. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 185–189 (2008)
Ward, W., Issar, S.: Recent Improvements in the CMU Spoken Language Understanding System. In: ARPA Human Language Technology Workshop, pp. 213–216 (1994)
Bohus, D., Rudnicky, A.: Integrating Multiple Knowledge Sources for Utterance-Level Confidence Annotation in the Cmu Communicator Spoken Dialogue System. Technical report, Carnegie Mellon University (2002)
SWIFT: Small Footprint Text-to-Speech Synthesizer, http://www.cepstral.com/
Bohus, D.: Error Awareness and Recovery in Task-Oriented Spoken Dialogue Systems. Ph.D. thesis proposal, Carnegie Mellon University (2004)
Stoyanchev, S., Stent, A.: Predicting Concept Types in User Corrections in Dialogue. In: EACL Workshop SRSL, pp. 42–49 (2009)
Litman, D., Hirschberg, J., Swerts, M.: Characterizing and Predicting Corrections in Spoken Dialogue Systems. Computational Linguistics 32, 417–438 (2006)
Dix, A., Finlay, J., Abowd, G.D., Beale, R.: Human-Computer Interaction. Prentice Hall (2003)
Rieser, V., Lemon, O.: Using Machine Learning to Explore Human Multimodal Clarification Strategies. In: COLING/ACL 2006, pp. 659–666 (2006)
Skantze, G.: Exploring Human Error Recovery Strategies: Implications for Spoken Dialogue Systems Speech Communication. Special Issue on Speech Annotation and Corpus Tools 45, 207–359 (2005)
Rieser, V., Kruijff-Korbayová, I., Lemon, O.: A Corpus Collection and Annotation Framework for Learning Multimodal Clarification Strategies. In: Sixth SIGdial Workshop on Discourse and Dialogue, pp. 97–106 (2005)
Sherwani, J., Yu, D., Paek, T., Czerwinski, M., Acero, A.: Voicepedia: Towards Speech-Based Access to Unstructured Information. In: Interspeech 2007 (2007)
Gordon, J.B., Passonneau, R.J.: An Evaluation Framework for Natural Language Understanding in Spoken Dialogue Systems. In: Seventh International Conference on International Language Resources and Evaluation (LREC 2010). European Language Resources Association, ELRA (2010)
Passonneau, R., Epstein, S.L., Gordon, J.B.: Help Me Understand You: Addressing the Speech Recognition Bottleneck. In: AAAI Spring Symposium on Agents that Learn from Human Teachers. AAAI (2009)
Ligorio, T., Epstein, S.L., Passonneau, R.J., Gordon, J.B.: What You Did and Didn’t Mean: Noise, Context, and Human Skill. In: Cognitive Science - 2010 (2010)
Passonneau, R.J., Epstein, S.L., Gordon, J.B., Ligorio, T.: Seeing What You Said: How Wizards Use Voice Search Results. In: IJCAI 2009 Workshop on Knowledge and Reasoning in Practical Dialogue Systems. AAAI Press (2009)
Ratcliff, J.W., Metzener, D.: Pattern Matching: The Gestalt Approach. Dr. Dobb’s Journal (1988)
Bangalore, S., Boulllier, P., Nasr, A., Rambow, O., Sagot, B.: Mica: A Probabilistic Dependency Parser Based on Tree Insertion Grammars. In: NAACL HLT 2009 Companion Volume: Short Papers, pp. 185–188 (2009)
Sacks, H., Schegloff, E.A., Jefferson, G.: A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 50, 696–735 (1974)
Allen, J., Ferguson, G., Stent, A.: An Architecture for More Realistic Conversational Systems. In: 6th International Conference on Intelligent User Interfaces, pp. 1–8 (2001)
Skantze, G., Gustafson, J.: Attention and Interaction Control in a Human-Human-Computer Dialogue Setting. In: Tenth Annual Meeting of the Special Interest Group in Dialogue and Discourse (SIGdial 10), pp. 310–313 (2009)
Gordon, J.B., Passonneau, R.J., Epstein, S.L.: Helping Agents Help Their Users Despite Imperfect Speech Recognition. In: AAAI Symposium Help Me Help You: Bridging the Gaps in Human-Agent Collaboration (2011)
Gordon, J., Epstein, S.L., Passonneau, R.J.: Learning to Balance Grounding Rationales for Dialogue Systems. In: 12th SIGDial on Dialogue and Discourse (2011)
Cameron, D.: The Myth of Mars and Venus: Do Men and Women Really Speak Different Languages?, Oxford (2007)
Cameron, D.: Sex/Gender, Language and the New Biologism. Applied Linguistics 31, 173–192 (2010)
Skantze, G., Edlund, J.: Early Error Detection on Word Level. In: ISCA Tutorial and Research Workshop on Robustness Issues in Conversational Interaction (2004)
Ligorio, T.: Feature Selection for Error Detection and Recovery in Spoken Dialogue Systems. Ph.D. thesis, Computer Science, The Graduate Center of The City University of New York, New York (2011)
Cao, L., Gorodetsky, V., Mitkas, P.: Agent Mining: The Synergy of Agents and Data Mining. IEEE Intelligent Systems 24(3), 64–72 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Epstein, S.L., Passonneau, R., Ligorio, T., Gordon, J. (2012). Data Mining to Support Human-Machine Dialogue for Autonomous Agents. In: Cao, L., Bazzan, A.L.C., Symeonidis, A.L., Gorodetsky, V.I., Weiss, G., Yu, P.S. (eds) Agents and Data Mining Interaction. ADMI 2011. Lecture Notes in Computer Science(), vol 7103. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27609-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-27609-5_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-27608-8
Online ISBN: 978-3-642-27609-5
eBook Packages: Computer ScienceComputer Science (R0)