Using Semantic Models for Robust Natural Language Human Robot Interaction

  • Emanuele BastianelliEmail author
  • Danilo Croce
  • Roberto Basili
  • Daniele Nardi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9336)


While robotic platforms are moving from industrial to consumer applications, the need of flexible and intuitive interfaces becomes more critical and the capability of governing the variability of human language a strict requirement. Grounding of lexical expressions, i.e. mapping words of a user utterance to the perceived entities of a robot operational scenario, is particularly critical. Usually, grounding proceeds by learning how to associate objects categorized in discrete classes (e.g. routes or sets of visual patterns) to linguistic expressions. In this work, we discuss how lexical mapping functions that integrate Distributional Semantics representations and phonetic metrics can be adopted to robustly automate the grounding of language expressions into the robotic semantic maps of a house environment. In this way, the pairing between words and objects into a semantic map facilitates the grounding without the need of an explicit categorization. Comparative measures demonstrate the viability of the proposed approach and the achievable robustness, quite crucial in operational robotic settings.


Semantic Model Automatic Speech Recognition Selection Policy Levenshtein Distance String Kernel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bastianelli, E., Bloisi, D.D., Capobianco, R., Cossu, F., Gemignani, G., Iocchi, L., Nardi, D.: On-line semantic mapping. In: ICAR 2013, November 2013Google Scholar
  2. 2.
    Bastianelli, E., Castellucci, G., Croce, D., Basili, R., Nardi, D.: Huric: a human robot interaction corpus. In: Proceedings of LREC 2014, Reykjavik, Iceland, May 2014Google Scholar
  3. 3.
    Bunescu, R., Mooney, R.J.: Subsequence kernels for relation extraction. In: Submitted to the Ninth Conference on Natural Language Learning (CoNLL-2005), July 2006Google Scholar
  4. 4.
    Chen, D.L., Mooney, R.J.: Learning to interpret natural language navigation instructions from observations. In: Proceedings of the 25th AAAI Conference on AI, pp. 859–865 (2011)Google Scholar
  5. 5.
    Connell, J., Marcheret, E., Pankanti, S., Kudoh, M., Nishiyama, R.: An extensible language interface for robot manipulation. In: Bach, J., Goertzel, B., Iklé, M. (eds.) AGI 2012. LNCS, vol. 7716, pp. 21–30. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  6. 6.
    Golland, D., Liang, P., Klein, D.: A game-theoretic approach to generating spatial descriptions. In: Proceedings of the EMNLP 2010, Stroudsburg, PA, USA, pp. 410–419 (2010)Google Scholar
  7. 7.
    Guadarrama, S., Riano, L., Golland, D., Gohring, D., Jia, Y., Klein, D., Abbeel, P., Darrell, T.: Grounding spatial relations for human-robot interaction. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, November 2013Google Scholar
  8. 8.
    Harnad, S.: The symbol grounding problem. Physica D: Nonlinear Phenomena 42(1–3), 335–346 (1990)CrossRefGoogle Scholar
  9. 9.
    Harris, Z.: Distributional structure. In: Katz, J.J., Fodor, J.A. (eds.) The Philosophy of Linguistics. Oxford University Press (1964)Google Scholar
  10. 10.
    Hemachandra, S., Kollar, T., Roy, N., Teller, S.: Following and interpreting narrated guided tours. In: Proceedings of the ICRA 2011, Shanghai, China, pp. 2574–2579 (2011)Google Scholar
  11. 11.
    Kollar, T., Tellex, S., Roy, D., Roy, N.: Toward understanding natural language directions. In: Proceedings of the 5th ACM/IEEE, HRI 2010, Piscataway, NJ, USA, pp. 259–266 (2010)Google Scholar
  12. 12.
    Kruijff, G., Zender, H., Jensfelt, P., Christensen, H.: Clarification dialogues in human-augmented mapping. In: Proceedings of the HRI 2006 (2006)Google Scholar
  13. 13.
    Levenshtein, V.: Binary codes capable of correcting deletions. Insertions and eversals. In: Soviet Physics Doklady, vol. 10, p. 707 (1966)Google Scholar
  14. 14.
    Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)zbMATHGoogle Scholar
  15. 15.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)Google Scholar
  16. 16.
    Miller, G.A.: Wordnet: A lexical database for english. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  17. 17.
    Nüchter, A., Hertzberg, J.: Towards semantic maps for mobile robots. Robot. Auton. Syst. 56(11), 915–926 (2008)CrossRefGoogle Scholar
  18. 18.
    Rosenthal, S., Biswas, J., Veloso, M.: An effective personal mobile robot agent through symbiotic human-robot interaction. In: Proceedings of AAMAS 2010, vol. 1, May 2010Google Scholar
  19. 19.
    Roy, D., yuh Hsiao, K., Gorniak, P., Mukherjee, N.: Grounding natural spoken language semantics in visual perception and motor controlGoogle Scholar
  20. 20.
    Sahlgren, M.: The Word-Space Model. Ph.D. thesis, Stockholm University (2006)Google Scholar
  21. 21.
    Schütze, H.: Word space. In: Advances in Neural Information Processing Systems, vol. 5, pp. 895–902. Morgan Kaufmann (1993)Google Scholar
  22. 22.
    Steels, L., Vogt, P.: Grounding adaptive language games in robotic agents. In: Proceedings of the Fourth European Conference on Artificial Life, pp. 474–482. MIT Press (1997)Google Scholar
  23. 23.
    Tellex, S., Kollar, T., Dickerson, S., Walter, M., Banerjee, A., Teller, S., Roy, N.: Approaching the symbol grounding problem with probabilistic graphical models. AI Magazine 32(4) (2011)Google Scholar
  24. 24.
    Topp, E.A.: Human-Robot Interaction and Mapping with a Service Robot: Human Augmented Mapping. Ph.D. thesis, Royal Institute of Technology, School of Computer Science and Communication (2008)Google Scholar
  25. 25.
    Winkler, W.E.: The state of record linkage and current research problems. Tech. rep., Statistical Research Division, U.S. Census Bureau (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Emanuele Bastianelli
    • 2
    Email author
  • Danilo Croce
    • 1
  • Roberto Basili
    • 1
  • Daniele Nardi
    • 3
  1. 1.DIIUniversity of Rome Tor VergataRomeItaly
  2. 2.DICIIUniversity of Rome Tor VergataRomeItaly
  3. 3.DIIAGSapienza University of RomeRomeItaly

Personalised recommendations