Abstract
Spoken dialogue system performance can vary widely for different users, as well for the same user during different dialogues. This paper presents the design and evaluation of an adaptive version of TOOT, a spoken dialogue system for retrieving online train schedules. Based on rules learned from a set of training dialogues, adaptive TOOT constructs a user model representing whether the user is having speech recognition problems as a particular dialogue progresses. Adaptive TOOT then automatically adapts its dialogue strategies based on this dynamically changing user model. An empirical evaluation of the system demonstrates the utility of the approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bell, L. and Gustafson, J.: 2000, Positive and Negative User Feedback in a Spoken Dialogue Corpus. In: Proc. 6th International Conference of Spoken Language Processing (ICSLP). Beijing, China, pp. 589–592.
Bouwman, A. G., Sturm, J. and Boves, L.: 1999, Incorporating confidence measures in the Dutch train timetable information system developed in the ARISE project. In: Proc. International Conference on Acoustics, Speech and Signal Processing, Vol. 1. Phoenix, pp. 493–496.
Chin, D. N.: 2001, Empirical Evaluation of User Models and User-Adapted Systems. User Modeling and User-Adapted Interaction, pp. 181–194.
Chu-Carroll, J.: 2000, MIMIC: An Adaptive Mixed Initiative Spoken Dialogue System for Information Queries. In: Proc. Applied Natural Language Processing (ANLP), pp. 97–104.
Chu-Carroll, J. and Nickerson, J. S.: 2000, Evaluating Automatic Dialogue Strategy Adaptation for a Stoken Dialogue System. In: Proc. 1st Conference of the North American Chapter of the Association for Coputational Linguistics (NAACL), pp. 202–209.
Cohen, P.: 1995, Empirical Methods for Artificial Intelligence. MIT Press, Boston.
Cohen, W.: 1996, Learning trees and rules with set-valued features. In: Proc. 13th National Conference on Artificial Intelligence (AAAI), pp. 709–716.
Danieli, M. and Gerbino, E.: 1996, Metrics for Evaluating Dialogue Strategies in a Spoken Language System. In: Proc. AAAI Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, pp. 34–39.
Hirasawa, J., Miyazaki, N., Nakano, M. and Aikawa, K.: 2000, New Feature Parameters For Detecting Misunderstandings in a Spoken Dialogue System. In: Proc. 6th International Conference of Spoken Language Processing (ICSLP), Vol. 2, Beijing, China, pp. 154–157.
Kamm, C., Litman, D. and Walker, M.: 1998, From Novice to Expert: The Effect of Tutorials on User Expertise with Spoken Dialogue Systems. In: Proc ICSLP, pp. 1211–1214.
Kamm, C., Narayanan, S., Dutton, D. and Ritenour, R.: 1997, Evaluating Spoken Dialog Systems for Telecommunication Services. In: Proc. European Conference on Speech Communication and Technology (EUROSPEECH), pp. 22–25.
Krahmer, E., Swerts, M., Theune, M. and Weegels, M.: 1999, Error Spotting in Human–Machine Interactions. In: Proc. European Conference on Speech Communication and Technology (EUROSPEECH), pp. 1423–1426.
Levin, E. and Pieraccini, R.: 1997, A Stochastic Model of Computer–Human Interaction for Learning Dialogue Strategies. In: Proc. European Conference on Speech Communication and Technology (EUROSPEECH), pp. 1883–1886.
Levow, G.-A: 1998, Characterizing and Recognizing Spoken Corrections in Human–Computer Dialogue. In: Proc. 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL), pp. 736–742.
Litman, D., Hirschberg, J. and Swerts, M.: 2000a, Predicting Automatic Speech Recognition Performance Using Prosodic Cues. In: Proc. 1st Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 218–225.
Litman, D., Pan, S. and Walker, M.: 1998, Evaluating Response Strategies in a Web-Based Spoken Dialogue Agent. In: Proc. ACL/COLING, pp. 780–786.
Litman, D. J.: 1998, Predicting Speech Recognition Performance from Dialogue Phenomena. Presented at the American Association for Artificial Intelligence Spring Symposium Series on Applying Machine Learning to Discourse Processing.
Litman, D. J., Kearns, M. S., Singh, S. and Walker, M. A.: 2000b, Automatic Optimization of Dialogue Management. In: Proc. of COLING 2000.
Litman, D. J. and Pan, S.: 1999, Empirically Evaluating an Adaptable Spoken Dialogue System. In: Proc. 7th International Conference on User Modeling (UM), pp. 55–64.
Litman, D. J., Walker, M. A. and Kearns, M. J.: 1999: Automatic Detection of Poor Speech Recognition at the Dialogue Level. In: Proc. 37th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 309–316.
Monge, P. and Cappella, J. (eds): 1980, Multivariate Techniques in Human Communication Research. Academic Press, New York.
Polifroni, J., Hirschman, L., Seneff, S. and Zue, V.: 1992, Experiments in Evaluating Interactive Spoken Language Systems. In: Proc. DARPA Speech and NL Workshop, pp. 28–33.
Shriberg, E., Wade, E. and Price, P.: 1992, Human–Machine Problem Solving Using Spoken Language Systems (SLS): Factors Affecting Performance and User Satisfaction. In: Proc. DARPA Speech and NL workshop, pp. 419–424.
Smith, R.W.: 1998, An Evaluation of Strategies for Selectively Verifying Utterance Meanings in Spoken Natural Lanauage Dialog. International Journal of Human–Computer Studies 48, 627–647.
Strachan, L., Anderson, J., Sneesby, M. and Evans, M.: 1997, Pragmatic UserModelling in a Commercial Software System. In: Proc. UM97, pp. 189–200.
van Zanten, G. V.: 1999, User Modelling in Adaptive Dialogue Management. In: Proc. European Conference on Speech Communication and Technology (EUROSPEECH), pp. 1183–1186.
Walker, M., Fromer, J. and Narayanan, S.: 1998a, Learning Optimal Dialogue Strategies. A Case Study of a Spoken Dialogue Agent for E-mail. In: Proc. 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL), pp. 1345–1352.
Walker, M., Hindle, D., Fromer, J., Fabbrizio,G. D. and Mestel, C.: 1997a, Evaluating Competing Agent Strategies for a Voice E-mail Agent. In: Proc. European Conference on Speech Communication and Technology (EUROSPEECH), pp. 22–25.
Walker, M., Langkilde, I., Wright, J., Gorin, A. and Litman, D.: 2000a. Learning to Predict Problematic Situations in a Spoken Dialogue System: Experiments with HowMay I Help You?. In: Proceedings of the North American Meeting of the Association for Computational Linguistics, pp. 210–217.
Walker, M., Litman, D., Kamm, C. and Abella, A.: 1997b, PARADISE: A General Framework for Evaluating Spoken Dialogue Agents. In: Proc. ACL/EACL, pp. 271–280.
Walker, M., Litman, D., Kamm, C. and Abella, A.:1998b, Evaluating Spoken Dialogue Agents with PARADISE Two Case Studies. Computer Speech and Language, 12(3), pp. 317–347.
Walker, M. A., Kamm, C. A. and Litman, D. J. 2000b, Towards Developing General Models of Usability with PARADISE. Natural Language Engineering: Special Issue on Best Practice in Spoken Dialogue Systems.
Weiss, S. M. and Kulikowski, C.: 1991, Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. San Mateo, CA: Morgan Kaufmann.
Zeljkovic, I.: 1996, Decoding Optimal State Sequences with Smooth State Likelihoods. In: Proc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 129–132.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Litman, D.J., Pan, S. Designing and Evaluating an Adaptive Spoken Dialogue System. User Modeling and User-Adapted Interaction 12, 111–137 (2002). https://doi.org/10.1023/A:1015036910358
Issue Date:
DOI: https://doi.org/10.1023/A:1015036910358