Sorry, I Didn’t Catch That!

Abstract

We present results from an extensive empirical analysis of non-understanding errors and ten non-understanding recovery strategies, based on a corpus of dialogues collected with a spoken dialogue system that handles conference room reservations. More specifically, the issues we investigate are: what are the main sources of non-understandings? What is the impact of these errors on global performance? How do various strategies for recovery from non-understandings compare to each other? What are the relationships between these strategies and subsequent user response types, and which response types are more likely to lead to successful recovery? Can dialogue performance be improved by using a smarter policy for engaging the non-understanding recovery strategies? If so, can we learn such a policy from data? Whenever available, we compare and contrast our results with other studies in the literature. Finally, we summarise the lessons learned and present our plans for future work inspired by this analysis.