Skip to main content
SpringerLink
Log in

Can I Do That? Discovering Domain Axioms Using Declarative Programming and Relational Reinforcement Learning

  • Conference paper
  • First Online:
Autonomous Agents and Multiagent Systems (AAMAS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10003))

Included in the following conference series:

Abstract

Robots deployed to assist humans in complex, dynamic domains need the ability to represent, reason with, and learn from, different descriptions of incomplete domain knowledge and uncertainty. This paper presents an architecture that integrates declarative programming and relational reinforcement learning to support cumulative and interactive discovery of previously unknown axioms governing domain dynamics. Specifically, Answer Set Prolog (ASP), a declarative programming paradigm, is used to represent and reason with incomplete commonsense domain knowledge. For any given goal, unexplained failure of plans created by inference in the ASP program is taken to indicate the existence of unknown domain axioms. The task of learning these axioms is formulated as a Reinforcement Learning problem, and decision-tree regression with a relational representation is used to generalize from specific axioms identified over time. The new axioms are added to the ASP-based representation for subsequent inference. We demonstrate and evaluate the capabilities of our architecture in two simulated domains: Blocks World and Simple Mario.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    We use the terms “robot”, “agent” and “learner” interchangeably in this paper.

  2. 2.

    We use the terms “ASP” and “CR-Prolog” interchangeably in this paper.

References

  1. Colaco, Z., Sridharan, M.: What happened and why? A mixed architecture for planning and explanation generation in robotics. In: Australasian Conference on Robotics and Automation (ACRA), 2–4 December 2015, Canberra, Australia (2015)

    Google Scholar 

  2. Zhang, S., Sridharan, M., Gelfond, M., Wyatt, J.: Towards an architecture for knowledge representation and reasoning in robotics. In: Beetz, M., Johnston, B., Williams, M.-A. (eds.) ICSR 2014. LNCS (LNAI), vol. 8755, pp. 400–410. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11973-1_41

    Google Scholar 

  3. Zhang, S., Sridharan, M., Wyatt, J.: Mixed logical inference and probabilistic planning for robots in unreliable worlds. IEEE Trans. Robot. 31(3), 699–713 (2015)

    Article  Google Scholar 

  4. Sridharan, M.: Towards an architecture for knowledge representation, reasoning and learning in human-robot collaboration. In: AAAI Spring Symposium on Enabling Computing Research in Socially Intelligent Human-Robot Interaction, 21–23 March 2016, Stanford, USA (2016)

    Google Scholar 

  5. Sridharan, M., Rainge, S.: Integrating reinforcement learning and declarative programming to learn causal laws in dynamic domains. In: Beetz, M., Johnston, B., Williams, M.-A. (eds.) ICSR 2014. LNCS (LNAI), vol. 8755, pp. 320–329. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11973-1_33

    Google Scholar 

  6. Sridharan, M., Gelfond, M.: Using knowledge representation and reasoning tools in the design of robots. In: IJCAI Workshop on Knowledge-Based Techniques for Problem Solving and Reasoning (KnowProS), 10 July 2016, New York, USA (2016)

    Google Scholar 

  7. Bai, H., Hsu, D., Lee, W.S.: Integrated perception and planning in the continuous space: a POMDP approach. Int. J. Robot. Res. 33(8), 1288–1302 (2014)

    Article  Google Scholar 

  8. Hoey, J., Poupart, P., Bertoldi, A., Craig, T., Boutilier, C., Mihailidis, A.: Automated handwashing assistance for persons with dementia using video and a partially observable markov decision process. Comput. Vis. Image Underst. 114(5), 503–519 (2010)

    Article  Google Scholar 

  9. Galindo, C., Fernandez-Madrigal, J.A., Gonzalez, J., Saffioti, A.: Robot task planning using semantic maps. Robot. Auton. Syst. 56(11), 955–966 (2008)

    Article  Google Scholar 

  10. Varadarajan, K.M., Vincze, M.: Ontological knowledge management framework for grasping and manipulation. In: IROS-2011 Workshop on Knowledge Representation for Autonomous Robots, 25 September 2011 (2011)

    Google Scholar 

  11. Gelfond, M., Kahl, Y.: Knowledge Representation, Reasoning and the Design of Intelligent Agents. Cambridge University Press, Cambridge (2014)

    Book  Google Scholar 

  12. Balduccini, M., Regli, W.C., Nguyen, D.N.: An ASP-based architecture for autonomous UAVs in dynamic environments: progress report. In: International Workshop on Non-Monotonic Reasoning (NMR), 17–19 July 2014, Vienna, Austria (2014)

    Google Scholar 

  13. Chen, X., Xie, J., Ji, J., Sui, Z.: Toward open knowledge enabling for human-robot interaction. J. Hum. Robot Interact. 1(2), 100–117 (2012)

    Google Scholar 

  14. Erdem, E., Patoglu, V.: Applications of action languages in cognitive robotics. In: Erdem, E., Lee, J., Lierler, Y., Pearce, D. (eds.) Correct Reasoning. LNCS, vol. 7265, pp. 229–246. Springer, Heidelberg (2012). doi:10.1007/978-3-642-30743-0_16

    Chapter  Google Scholar 

  15. Laird, J.E.: Extending the soar cognitive architecture. In: International Conference on Artificial General Intelligence, 1–3 March 2008, Memphis, USA (2008)

    Google Scholar 

  16. Talamadupula, K., Benton, J., Kambhampati, S., Schermerhorn, P., Scheutz, M.: Planning for human-robot teaming in open worlds. ACM Trans. Intell. Syst. Technol. 1(2), 14:1–14:24 (2010)

    Article  Google Scholar 

  17. Kaelbling, L., Lozano-Perez, T.: Integrated task and motion planning in belief space. Int. J. Robot. Res. 32(9–10), 1194–1227 (2013)

    Article  Google Scholar 

  18. Saribatur, Z., Erdem, E., Patoglu, V.: Cognitive factories with multiple teams of heterogeneous robots: hybrid reasoning for optimal feasible global plans. In: International Conference on Intelligent Robots and Systems, Chicago, USA, pp. 2923–2930 (2014)

    Google Scholar 

  19. Hanheide, M., Gretton, C., Dearden, R., Hawes, N., Wyatt, J., Pronobis, A., Aydemir, A., Gobelbecker, M., Zender, H.: Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour. In: International Joint Conference on Artificial Intelligence (IJCAI), 16–22 July 2011, Barcelona, Spain (2011)

    Google Scholar 

  20. Hanheide, M., Gobelbecker, M., Horn, G., Pronobis, A., Sjoo, K., Jensfelt, P., Gretton, C., Dearden, R., Janicek, M., Zender, H., Kruijff, G.J., Hawes, N., Wyatt, J.: Robot task planning and explanation in open and uncertain worlds. Artificial Intelligence (2015). http://dx.doi.org/10.1016/j.artint.2015.08.008

  21. Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1–2), 107–136 (2006)

    Article  Google Scholar 

  22. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: probabilistic models with unknown objects. In: Getoor, L., Taskar, B. (eds.) Statistical Relational Learning. MIT Press, Cambridge (2006)

    Google Scholar 

  23. Halpern, J.Y.: Reasoning about Uncertainty. MIT Press, Cambridge (2003)

    MATH  Google Scholar 

  24. Sanner, S., Kersting, K.: Symbolic dynamic programming for first-order POMDPs. In: AAAI Conference on Artificial Intelligence, 11–15 July 2010, Atlanta, USA, pp. 1140–1146 (2010)

    Google Scholar 

  25. Baral, C., Gelfond, M., Rushton, N.: Probabilistic reasoning with answer sets. Theory Pract. Logic Program. 9(1), 57–144 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lee, J., Wang, Y.: A probabilistic extension of the stable model semantics. In: AAAI Spring Symposium on Logical Formalizations of Commonsense Reasoning, March 2015)

    Google Scholar 

  27. Sutton, R.L., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  28. Dzeroski, S., Raedt, L.D., Driessens, K.: Relational reinforcement learning. Mach. Learn. 43, 7–52 (2001)

    Article  MATH  Google Scholar 

  29. Tadepalli, P., Givan, R., Driessens, K.: Relational reinforcement learning: an overview. In: Relational Reinforcement Learning Workshop at the International Conference on Machine Learning (2004)

    Google Scholar 

  30. Blockeel, H., Raedt, L.D.: Top-down induction of first-order logical decision trees. Artif. Intell. 101(1–2), 285–297 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  31. Otero, R.P.: Induction of the effects of actions by monotonic methods. In: Horváth, T., Yamamoto, A. (eds.) ILP 2003. LNCS (LNAI), vol. 2835, pp. 299–310. Springer, Heidelberg (2003). doi:10.1007/978-3-540-39917-9_20

    Chapter  Google Scholar 

  32. Sridharan, M., Gelfond, M., Zhang, S., Wyatt, J.: A refinement-based architecture for knowledge representation and reasoning in robotics. Technical report, Unrefereed CoRR abstract, August 2015. http://arxiv.org/abs/1508.03891

  33. Balduccini, M., Gelfond, M.: Logic programs with consistency-restoring rules. In: AAAI Spring Symposium on Logical Formalization of Commonsense Reasoning, pp. 9–18 (2003)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the US Office of Naval Research Science of Autonomy award N00014-13-1-0766. All opinions and conclusions in this paper are those of the authors alone.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, The University of Auckland, Auckland, New Zealand

    Mohan Sridharan, Prashanth Devarakonda & Rashmica Gupta

Authors
  1. Mohan Sridharan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prashanth Devarakonda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rashmica Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohan Sridharan .

Editor information

Editors and Affiliations

  1. Campus de la UAB, IIIA-CSIC, Bellaterra, Spain

    Nardine Osman

  2. Campus de la UAB, IIIA-CSIC, Cerdanyola, Spain

    Carles Sierra

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Sridharan, M., Devarakonda, P., Gupta, R. (2016). Can I Do That? Discovering Domain Axioms Using Declarative Programming and Relational Reinforcement Learning. In: Osman, N., Sierra, C. (eds) Autonomous Agents and Multiagent Systems. AAMAS 2016. Lecture Notes in Computer Science(), vol 10003. Springer, Cham. https://doi.org/10.1007/978-3-319-46840-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-46840-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-46839-6

  • Online ISBN: 978-3-319-46840-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation