Using Probabilistic Feature Matching to Understand Spoken Descriptions

  • Ingrid Zukerman
  • Enes Makalic
  • Michael Niemann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5360)


We describe a probabilistic reference disambiguation mechanism developed for a spoken dialogue system mounted on an autonomous robotic agent. Our mechanism performs probabilistic comparisons between features specified in referring expressions (e.g. size and colour) and features of objects in the domain. The results of these comparisons are combined using a function weighted on the basis of the specified features. Our evaluation shows high reference resolution accuracy across a range of spoken referring expressions.


Lexical Item Match Probability Probabilistic Feature Parse Tree Dialogue System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sowa, J.: Conceptual Structures: Information Processing in Mind and Machine. Addison-Wesley, Reading (1984)zbMATHGoogle Scholar
  2. 2.
    Zukerman, I., Makalic, E., Niemann, M., George, S.: A probabilistic approach to the interpretation of spoken utterances. In: Ho, T.-B., Zhou, Z.-H. (eds.) PRICAI 2008. LNCS (LNAI), vol. 5351, pp. 581–592. Springer, Heidelberg (2008)Google Scholar
  3. 3.
    Makihara, Y., Takizawa, M., Shirai, I., Miura, J., Shimada, N.: Object recognition supported by user interaction for service robots. In: Proceedings of the 16th International Conference on Pattern Recognition, Quebec, Canada, vol. 3, pp. 561–564 (2002)Google Scholar
  4. 4.
    Dale, R., Reiter, E.: Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science 18(2), 233–263 (1995)CrossRefGoogle Scholar
  5. 5.
    Wyatt, J.: Planning clarification questions to resolve ambiguous references to objects. In: Proceedings of the 4th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, Edinburgh, Scotland, pp. 16–23 (2005)Google Scholar
  6. 6.
    Leacock, C., Chodorow, M.: Combining local context and WordNet similarity for word sense identification. In: Fellbaum, C. (ed.) WordNet: An Electronic Lexical Database, pp. 265–285. MIT Press, Cambridge (1998)Google Scholar
  7. 7.
    Pedersen, T., Patwardhan, S., Michelizzi, J.: WordNet: Similarity – measuring the relatedness of concepts. In: AAAI 2004 – Proceedings of the 19th National Conference on Artificial Intelligence, San Jose, California, pp. 25–29 (2004)Google Scholar
  8. 8.
    Puzicha, J., Buhmann, J., Rubner, Y., Tomasi, C.: Empirical evaluation of dissimilarity measures for color and texture. In: Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, vol. 2, pp. 1165–1172 (1999)Google Scholar
  9. 9.
    Kelleher, J.: Attention driven reference resolution in multimodal contexts. Artificial Intelligence Review 25, 21–35 (2006)CrossRefGoogle Scholar
  10. 10.
    Potamianos, G., Neti, C.: Stream confidence estimation for audio-visual speech recognition. In: ICSLP 2000 Proceedings, Beijing, China, vol. 3, pp. 746–749 (2000)Google Scholar
  11. 11.
    Carletta, J.: Assessing agreement on classification tasks: The Kappa statistic. Computational Linguistics 22(2), 249–254 (1996)Google Scholar
  12. 12.
    Ang, J., Dhillon, R., Krupski, A., Shriberg, E., Stolcke, A.: Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: ICSLP 2002 Proceedings, Denver, Colorado, pp. 2037–2040 (2002)Google Scholar
  13. 13.
    Siddharthan, A., Copestake, A.: Generating referring expressions in open domains. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, pp. 407–414 (2004)Google Scholar
  14. 14.
    Pfleger, N., Alexandersson, J., Becker, T.: A robust and generic discourse model for multimodal dialogue. In: Proceedings of the 3rd IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, Acapulco, Mexico (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Ingrid Zukerman
    • 1
  • Enes Makalic
    • 1
  • Michael Niemann
    • 1
  1. 1.Faculty of Information TechnologyMonash UniversityClaytonAustralia

Personalised recommendations