From Natural Language Instructions to Structured Robot Plans

  • Mihai Pomarlan
  • Sebastian Koralewski
  • Michael Beetz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10505)


Research into knowledge acquisition for robotic agents has looked at interpreting natural language instructions meant for humans into robot-executable programs; however, the ambiguities of natural language remain a challenge for such “translations”. In this paper, we look at a particular sort of ambiguity: the control flow structure of the program described by the natural language instruction. It is not always clear, when more conditional statements appear in a natural language instruction, which of the conditions are to be thought of as alternative options in the same test, and which belong to a code branch triggered by a previous conditional. We augment a system which uses probabilistic reasoning to identify the meaning of the words in a sentence with reasoning about action preconditions and effects in order to filter out non-sensical code structures. We test our system with sample instruction sheets inspired from analytical chemistry.


  1. 1.
    Nyga, D., Beetz, M.: Cloud-based probabilistic knowledge services for instruction interpretation. In: International Symposium of Robotics Research (ISRR), Italy, Sestri Levante (Genoa) (2015)Google Scholar
  2. 2.
    Misra, D.K., Sung, J., Lee, K., Saxena, A.: Tell me dave: context-sensitive grounding of natural language to manipulation instructions. In: Proceedings of Robotics: Science and Systems, Berkeley, USA, July 2014Google Scholar
  3. 3.
    Abbott, B.: Conditionals in English and fopl. In: Shu, D., Turner, K., (eds.) Contrasting Meanings in Languages of the East and West, pp. 579–606. Peter Lang, Oxford (2010)Google Scholar
  4. 4.
    Rothschild, D.: Conditionals and propositions in semantics. J. Philos. Logic 44(6), 781 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Narayanan, R., Liu, B., Choudhary, A.: Sentiment analysis of conditional sentences. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 180–189. Association for Computational Linguistics (2009)Google Scholar
  6. 6.
    Dufour-Lussier, V., Le Ber, F., Lieber, J., Nauer, E.: Automatic case acquisition from texts for process-oriented case-based reasoning. Inf. Syst. 40, 153–167 (2014)CrossRefGoogle Scholar
  7. 7.
    Schumacher, P., Minor, M.: Extracting control-flow from text. In: IRI, pp. 203–210. IEEE (2014)Google Scholar
  8. 8.
    Bos, J., Oka, T.: A spoken language interface with a mobile robot. Artif. Life Robot. 11(1), 42–47 (2007)CrossRefGoogle Scholar
  9. 9.
    Misra, D.K., Tao, K., Liang, P., Saxena, A.: Environment-driven lexicon induction for high-level instructions. In: ACL (1), pp. 992–1002 (2015)Google Scholar
  10. 10.
    Eppe, M., Trott, S., Feldman, J.: Exploiting deep semantics and compositionality of natural language for human-robot-interaction. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 731–738. IEEE (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mihai Pomarlan
    • 1
  • Sebastian Koralewski
    • 1
  • Michael Beetz
    • 1
  1. 1.Institute for Artificial IntelligenceUniversität BremenBremenGermany

Personalised recommendations