Skip to main content
SpringerLink
Log in

Extracting Planning Operators from Instructional Texts for Behaviour Interpretation

  • Conference paper
  • First Online:
KI 2018: Advances in Artificial Intelligence (KI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11117))

Abstract

Recent attempts at behaviour understanding through language grounding have shown that it is possible to automatically generate planning models from instructional texts. One drawback of these approaches is that they either do not make use of the semantic structure behind the model elements identified in the text, or they manually incorporate a collection of concepts with semantic relationships between them. To use such models for behaviour understanding, however, the system should also have knowledge of the semantic structure and context behind the planning operators. To address this problem, we propose an approach that automatically generates planning operators from textual instructions. The approach is able to identify various hierarchical, spatial, directional, and causal relations between the model elements. This allows incorporating context knowledge beyond the actions being executed. We evaluated the approach in terms of correctness of the identified elements, model search complexity, model coverage, and similarity to handcrafted models. The results showed that the approach is able to generate models that explain actual tasks executions and the models are comparable to handcrafted models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that regression usually reflects correlation. Granger, however, argued that causality in economics could be tested for by measuring the ability to predict the future values of a time series using prior values of another time series. As the question of “true causality” is philosophical, the Granger causality test assumes that one thing preceding another can be used as evidence of causation.

  2. 2.

    https://www.wikihow.com/Main-Page.

  3. 3.

    http://kitchen.cs.cmu.edu/.

  4. 4.

    These descriptions have been generated based on the observed in the video log behaviour.

  5. 5.

    Note that we did not compare the identified causal relations. This is because implicit causal relations are a subject of interpretation. For that reason, we consider the relations are correctly identified if the model is able to explain the given plan.

  6. 6.

    In the case where the number of elements was the same for both the human annotator and our tool, the discovered elements were the same in both cases.

References

  1. Babeş-Vroman, M., et al.: Learning to interpret natural language instructions. In: Proceedings of the Workshop on Semantic Interpretation in an Actionable Context, Stroudsburg, PA, USA, pp. 1–6 (2012)

    Google Scholar 

  2. Baker, C., Saxe, R., Tenenbaum, J.: Action understanding as inverse planning. Cognition 113(3), 329–349 (2009)

    Article  Google Scholar 

  3. Benotti, L., Lau, T., Villalba, M.: Interpreting natural language instructions using language, vision, and behavior. ACM Trans. Interact. Intell. Syst. 4(3), 13:1–13:22 (2014)

    Article  Google Scholar 

  4. Branavan, S., Kushman, N., Lei, T., Barzilay, R.: Learning high-level planning from text. In: Proceedings of the Annual Meeting of Association for Computational Linguistics, Stroudsburg, PA, USA, pp. 126–135 (2012)

    Google Scholar 

  5. Branavan, S., Zettlemoyer, L., Barzilay, R.: Reading between the lines: learning to map high-level instructions to commands. In: Proceedings of the Annual Meeting of Association for Computational Linguistics, Stroudsburg, PA, USA, pp. 1268–1277 (2010)

    Google Scholar 

  6. Chen, D., Mooney, R.: Learning to interpret natural language navigation instructions from observations. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 859–865, August 2011

    Google Scholar 

  7. Diebold, F., Witman, K., Hanseman, D., Lysne, L., Moore, T.: Elements of Forecasting, 2nd edn. Cengage Learning, Boston (2000)

    Google Scholar 

  8. Goldwasser, D., Roth, D.: Learning from natural instructions. Mach. Learn. 94(2), 205–232 (2014)

    Article  MathSciNet  Google Scholar 

  9. Granger, C.: Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3), 424–438 (1969)

    Article  Google Scholar 

  10. Kirste, T., Krüger, F.: CCBM-a tool for activity recognition using computational causal behavior models. Technical report CS-01-12. Institut für Informatik, Universität Rostock, Rostock, Germany, May 2012

    Google Scholar 

  11. Kollar, T., Tellex, S., Roy, D., Roy, N.: Grounding verbs of motion in natural language commands to robots. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) Experimental Robotics, vol. 79, pp. 31–47. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-28572-1_3

    Chapter  Google Scholar 

  12. Krüger, F., Nyolt, M., Yordanova, K., Hein, A., Kirste, T.: Computational state space models for activity and intention recognition. A feasibility study. PLoS ONE 9(11), e109381 (2014)

    Article  Google Scholar 

  13. Li, X., Mao, W., Zeng, D., Wang, F.-Y.: Automatic construction of domain theory for attack planning. In: IEEE International Conference on Intelligence and Security Informatics, pp. 65–70, May 2010

    Google Scholar 

  14. Lindsay, A., Read, J., Ferreira, J., Hayton, T., Porteous, J., Gregory, P.: Framer: planning models from natural language action descriptions. In: International Conference on Automated Planning and Scheduling (2017)

    Google Scholar 

  15. Miller, G.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  16. Nguyen, T.A., Kambhampati, S., Do, M.: Synthesizing robust plans under incomplete domain models. In: Advances in Neural Information Processing Systems 26, pp. 2472–2480. Curran Associates Inc. (2013)

    Google Scholar 

  17. Philipose, M., Fishkin, K., Perkowitz, M., Patterson, D., Fox, D., Kautz, H., Hahnel, D.: Inferring activities from interactions with objects. IEEE Pervasive Comput. 3(4), 50–57 (2004)

    Article  Google Scholar 

  18. Ramirez, M., Geffner, H.: Goal recognition over POMDPs: inferring the intention of a POMDP agent. In: Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI 2011, Barcelona, Spain, vol. 3, pp. 2009–2014 (2011)

    Google Scholar 

  19. Sil, A., Yates, A.: Extracting strips representations of actions and events. In: Recent Advances in Natural Language Processing, Hissar, Bulgaria, pp. 1–8, September 2011

    Google Scholar 

  20. Tenorth, M., Nyga, D., Beetz, M.: Understanding and executing instructions for everyday manipulation tasks from the world wide web. In: IEEE International Conference on Robotics and Automation, pp. 1486–1491, May 2010

    Google Scholar 

  21. Veloso, M., Perez, A., Carbonell, J.: Nonlinear planning with parallel resource allocation. In: Proceedings of the DARPA Workshop of Innovative Approaches to Planning, Scheduling and Control (1990)

    Google Scholar 

  22. Vogel, A., Jurafsky, D.: Learning to follow navigational directions. In: Proceedings of the Annual Meeting of Association for Computational Linguistics, Stroudsburg, PA, USA, pp. 806–814 (2010)

    Google Scholar 

  23. Webber, B., Badler, N., Eugenio, B., Geib, C., Levison, L., Moore, M.: Instructions, intentions and expectations. Artif. Intell. 73(1), 253–269 (1995)

    Article  Google Scholar 

  24. Ye, J., Dobson, S., McKeever, S.: Situation identification techniques in pervasive computing: a review. Pervasive Mob. Comput. 8(1), 36–66 (2012)

    Article  Google Scholar 

  25. Yordanova, K.: Discovering causal relations in textual instructions. In: Recent Advances in Natural Language Processing, Hissar, Bulgaria, pp. 714–720, September 2015

    Google Scholar 

  26. Yordanova, K.: Automatic generation of situation models for plan recognition problems. In: Proceedings of the International Conference Recent Advances in Natural Language Processing, Varna, Bulgaria, pp. 823–830. INCOMA Ltd., September 2017

    Google Scholar 

  27. Yordanova, K.: A simple model for improving the performance of the Stanford Parser for action detection in textual instructions. In: Proceedings of the International Conference Recent Advances in Natural Language Processing, Varna, Bulgaria, pp. 831–838. INCOMA Ltd., September 2017

    Google Scholar 

  28. Zhang, Z., Webster, P., Uren, V., Varga, A., Ciravegna, F.: Automatically extracting procedural knowledge from instructional texts using natural language processing. In: Proceedings of the International Conference on Language Resources and Evaluation, Istanbul, Turkey, pp. 520–527, May 2012

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Rostock, 18059, Rostock, Germany

    Kristina Yordanova

Authors
  1. Kristina Yordanova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristina Yordanova .

Editor information

Editors and Affiliations

  1. TU Berlin, Berlin, Germany

    Frank Trollmann

  2. TU Dresden, Dresden, Germany

    Anni-Yasmin Turhan

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yordanova, K. (2018). Extracting Planning Operators from Instructional Texts for Behaviour Interpretation. In: Trollmann, F., Turhan, AY. (eds) KI 2018: Advances in Artificial Intelligence. KI 2018. Lecture Notes in Computer Science(), vol 11117. Springer, Cham. https://doi.org/10.1007/978-3-030-00111-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00111-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00110-0

  • Online ISBN: 978-3-030-00111-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation