Skip to main content
SpringerLink
Log in

Continual planning and acting in dynamic multiagent environments

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

In order to behave intelligently, artificial agents must be able to deliberatively plan their future actions. Unfortunately, realistic agent environments are usually highly dynamic and only partially observable, which makes planning computationally hard. For most practical purposes this rules out planning techniques that account for all possible contingencies in the planning process. However, many agent environments permit an alternative approach, namely continual planning, i.e. the interleaving of planning with acting and sensing. This paper presents a new principled approach to continual planning that describes why and when an agent should switch between planning and acting. The resulting continual planning algorithm enables agents to deliberately postpone parts of their planning process and instead actively gather missing information that is relevant for the later refinement of the plan. To this end, the algorithm explictly reasons about the knowledge (or lack thereof) of an agent and its sensory capabilities. These concepts are modelled in the planning language (MAPL). Since in many environments the major reason for dynamism is the behaviour of other agents, MAPL can also model multiagent environments, common knowledge among agents, and communicative actions between them. For Continual Planning, MAPL introduces the concept of of assertions, abstract actions that substitute yet unformed subplans. To evaluate our continual planning approach empirically we have developed MAPSIM, a simulation environment that automatically builds multiagent simulations from formal MAPL domains. Thus, agents can not only plan, but also execute their plans, perceive their environment, and interact with each other. Our experiments show that, using continual planning techniques, deliberate action planning can be used efficiently even in complex multiagent environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ambros-Ingerson, J. A., & Steel, S. (1988). Integrating planning, execution and monitoring. In Proceedings of AAAI-88 (pp. 83–88), Saint Paul, MI.

  2. Bäckström C., Nebel B. (1995) Complexity results for SAS+ planning. Computational Intelligence 11: 625–655

    Article  MathSciNet  Google Scholar 

  3. Bäckström C. (1998) Computational aspects of reordering plans. JAIR 9: 99–137

    Google Scholar 

  4. Blaylock, N., Allen, J., & Ferguson, G. (2003). Managing communicative intentions with collaborative problem solving. In Current and new directions in dialogue. Kluwer.

  5. Boutilier C., Brafman R. (2001) Partial order planning with concurrent interacting actions. JAIR 14: 105–136

    MATH  Google Scholar 

  6. Brenner, M. (2005). Planning for multiagent environments: From individual perceptions to coordinated execution. In Wsh. Multiagent Planning and Scheduling, ICAPS ’05, Monterey, USA.

  7. Brenner, M. (2007). Situation-aware interpretation, planning and execution of user commands by autonomous robots. In Proceedings of IEEE RO-MAN 2007.

  8. Brenner, M., Hawes, N. A., Kelleher, J. D., & Wyatt, J. (2007). Mediating between qualitative and quantitative representations for task-orientated human–robot interaction. In Proceedings of IJCAI 2007.

  9. Brenner, M., & Kruijff-Korbayová, I. (2008). A continual multiagent planning approach to situated dialogue. In Proceedings of the 12th workshop on the semantics and pragmatics of dialogue (semdial), London, UK.

  10. Clark, H. H., & Marshall, C. R. (1981). Definite reference and mutual knowledge. In Elements of discourse understanding. Cambridge University Press.

  11. Clement, B., & Barrett, A. (2003). Continual coordination through shared activities. In Proceedings of AAMAS ’03.

  12. Clement, B., & Durfee, E. (1999). Top-down search for coordinating the hierarchical plans of multiple agents. In Proceedings of AGENTS ’99.

  13. Cox, J. S., & Durfee, E. H. (2005). An efficient algorithm for multiagent plan coordination. In Proceedings of AAMAS ’05.

  14. DesJardins M., Durfee E., Ortiz C. Jr., Wolverton M. (1999) A survey of research in distributed, continual planning. The AI Magazine 20(4): 13–22

    Google Scholar 

  15. DesJardins, M., & Wolverton, M. (1999). Coordinating a distributed planning system. The AI Magazine, 20(4).

  16. Durfee E., Montgomery T. (1991) Coordination as distributed search in hierarchical behavior space. IEEE Transactions on Systems, Man, and Cybernetics 21(6): 1363–1378

    Article  Google Scholar 

  17. Durfee E.H. (1999) Distributed continual planning for unmanned ground vehicle teams. AI Magazine 20(4): 55–61

    Google Scholar 

  18. Edelkamp, S., & Hoffmann, J. (2004). PDDL2.2: The language for the classical part of the 4th international planning competition. Technical Report 195, Albert-Ludwigs-Universität, Institut für Informatik, Freiburg, Germany.

  19. Erol K., Hendler J., Nau D. (1996) Complexity results for hierarchical task-network planning. Annals of Mathematics and Artificial Intelligence 18: 69–93

    Article  MATH  MathSciNet  Google Scholar 

  20. Etzioni O., Golden K., Weld D.S. (1997) Sound and efficient closed-world reasoning for planning. Artificial Intelligence 89(1–2): 113–148

    Article  MATH  MathSciNet  Google Scholar 

  21. Etzioni, O., Hanks, S., Weld, D., Draper, D., Lesh, N., & Williamson, M. (1992). An approach to planning with incomplete information. In Proceedings of KR-92 (pp. 115–125).

  22. Fagin, R., Halpern, J. Y., Moses, Y., & Vardi, M. Y. (1995). Reasoning about knowledge. MIT Press.

  23. Fox M., Long D. (2003) PDDL 2.1: An extension to PDDL for expressing temporal planning domains. JAIR 20: 61–124

    MATH  Google Scholar 

  24. Geffner H. (2000) Functional STRIPS: A more flexible language for planning and problem solving. In: Minker J. (eds) Logic-based artificial intelligence. Kluwer, Dordrecht, Holland

    Google Scholar 

  25. Golden, K. (1998). Leap before you look: Information gathering in the puccini planner. In Proceedings of AIPS-98 (pp. 70–77).

  26. Golden, K., & Weld, D. (1996). Representing sensing actions: The middle ground revisited. In Proceedings of KR ’96.

  27. Grosz, B. J., & Kraus, S. (1996). Collaborative plans for complex group action. Artificial Intelligence, 86.

  28. Helmert M. (2006) The Fast Downward planning system. Journal of Artificial Intelligence Research 26: 191–246

    Article  MATH  Google Scholar 

  29. Hobbs, J.R., Moore, R.C. (eds) (1985) Formal theories of the commonsense world. Ablex, Norwood, NJ

    Google Scholar 

  30. Hoffmann, J., & Brafman, R. (2005). Contingent planning via heuristic forward search with implicit belief states. In S. Biundo, K. L. Myers, & K. Rajan (Eds.), ICAPS (pp. 71–80). AAAI.

  31. Hoffmann J., Brafman R.I. (2006) Conformant planning via heuristic forward search: A new approach. Artificial Intelligence 170: 507–541

    Article  MATH  MathSciNet  Google Scholar 

  32. Kambhampati, S. (2007). Model-lite planning for the web age masses: The challenges of planning with incomplete and evolving domain models. In Proceedings of AAAI-07.

  33. Knoblock C.A. (1995) Planning, executing, sensing, and replanning for information gathering. In: Mellish C. (eds) Proceedings of the fourteenth international joint conference on artificial intelligence. Morgan Kaufmann, San Francisco, pp 1686–1693

    Google Scholar 

  34. Kruijff, G.-J., & Brenner, M. (2007). Modelling spatio-temporal comprehension in situated human–robot dialogue as reasoning about intentions and plans. In AAAI spring symposium on intentions in intelligent systems, Stanford, CA.

  35. Levesque, H. J. (1996). What is planning in the presence of sensing? In Proceedings of AAAI-96 (pp. 1139–1146). MIT Press.

  36. Lewis D. (1969) Convention. A philosophical study. Harvard University Press, Cambridge, MA

    Google Scholar 

  37. Littman M., Goldsmith J., Mundhenk M. (1998) The computational complexity of probabilistic planning. JAIR 9: 1–36

    MATH  MathSciNet  Google Scholar 

  38. Lochbaum K.E. (1998) A collaborative planning model of intentional structure. Computational Linguistics 24: 525–572

    Google Scholar 

  39. Lynch N. (1996) Distributed algorithms. Morgan Kaufmann, San Francisco, CA

    MATH  Google Scholar 

  40. McDermott, D. (1998). PDDL—the planning domain definition language. Technical Report TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control.

  41. Myers, K. L. (1999). Cpef: A continuous planning and execution framework. The AI Magazine, 20(4).

  42. Nau D., Cao Y., Lotem A., Munoz-Avila H. (1999) SHOP: Simple hierarchical ordered planner. In: Dean T. (eds) Proceedings of the 16th international joint conference on artificial intelligence (IJCAI-99). Morgan Kaufmann, Stockholm, Sweden

    Google Scholar 

  43. Petrick, R., & Bacchus, F. (2002). A knowledge-based approach to planning with incomplete information and sensing. In Proceedings of AIPS-02.

  44. Petrick, R. P. A., & Bacchus, F. (2004). Extending the knowledge-based approach to planning with incomplete information and sensing. In Proceedings of ICAPS 2004 (pp. 2–11).

  45. Ram, A., Leake, D.B. (eds) (1995) Goal-driven learning. MIT Press, Cambridge, MA

    MATH  Google Scholar 

  46. Rich C., Sidner C.L. (1998) Collagen: A collaboration manager for software interface agents. UMUAI 8(3–4): 315–350

    Google Scholar 

  47. Rintanen J. (1999) Constructing conditional plans by a theorem-prover. JAIR 10: 323–352

    MATH  Google Scholar 

  48. Russell S., Norvig P. (2003) Artificial intelligence: A modern approach, 2nd edn. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  49. Sadek, M. D. (1991). Dialogue acts are rational plans. In Proceedings of the ESCA/ETR workshop on multi-modal dialogue. Italy: Maratea.

  50. Smith, D. E., & Weld, D. S. (1998). Conformant graphplan. In AAAI/IAAI (pp. 889–896).

  51. Thiebaux, S., Hoffmann, J., & Nebel, B. (2003). In defense of axioms in PDDL. In Proceedings of IJCAI.

  52. Traum, D. (1994). A computational model of grounding in natural language conversation. PhD thesis, University of Rochester.

  53. Weld, D. S., Anderson, C. R., & Smith, D. E. (1998). Extending graphplan to handle uncertainty and sensing actions. In AAAI/IAAI (pp. 897–904).

  54. Yang, Q. (1997). Intelligent planning: A decomposition and abstraction based approach. Springer.

  55. Yokoo, M., & Hirayama, K. (2000). Algorithms for distributed constraint satisfaction: A review. Autonomous Agents and Multi-Agent Systems, 3(2).

Download references

Author information

Authors and Affiliations

  1. Institute for Computer Science, Albert-Ludwigs-Universität, Freiburg, Germany

    Michael Brenner & Bernhard Nebel

Authors
  1. Michael Brenner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Nebel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Brenner.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brenner, M., Nebel, B. Continual planning and acting in dynamic multiagent environments. Auton Agent Multi-Agent Syst 19, 297–331 (2009). https://doi.org/10.1007/s10458-009-9081-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-009-9081-1

Keywords

Search

Navigation