Autonomous Agents and Multi-Agent Systems

, Volume 19, Issue 3, pp 297–331 | Cite as

Continual planning and acting in dynamic multiagent environments

  • Michael Brenner
  • Bernhard Nebel


In order to behave intelligently, artificial agents must be able to deliberatively plan their future actions. Unfortunately, realistic agent environments are usually highly dynamic and only partially observable, which makes planning computationally hard. For most practical purposes this rules out planning techniques that account for all possible contingencies in the planning process. However, many agent environments permit an alternative approach, namely continual planning, i.e. the interleaving of planning with acting and sensing. This paper presents a new principled approach to continual planning that describes why and when an agent should switch between planning and acting. The resulting continual planning algorithm enables agents to deliberately postpone parts of their planning process and instead actively gather missing information that is relevant for the later refinement of the plan. To this end, the algorithm explictly reasons about the knowledge (or lack thereof) of an agent and its sensory capabilities. These concepts are modelled in the planning language (MAPL). Since in many environments the major reason for dynamism is the behaviour of other agents, MAPL can also model multiagent environments, common knowledge among agents, and communicative actions between them. For Continual Planning, MAPL introduces the concept of of assertions, abstract actions that substitute yet unformed subplans. To evaluate our continual planning approach empirically we have developed MAPSIM, a simulation environment that automatically builds multiagent simulations from formal MAPL domains. Thus, agents can not only plan, but also execute their plans, perceive their environment, and interact with each other. Our experiments show that, using continual planning techniques, deliberate action planning can be used efficiently even in complex multiagent environments.


Multiagent planning Continual planning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ambros-Ingerson, J. A., & Steel, S. (1988). Integrating planning, execution and monitoring. In Proceedings of AAAI-88 (pp. 83–88), Saint Paul, MI.Google Scholar
  2. 2.
    Bäckström C., Nebel B. (1995) Complexity results for SAS+ planning. Computational Intelligence 11: 625–655CrossRefMathSciNetGoogle Scholar
  3. 3.
    Bäckström C. (1998) Computational aspects of reordering plans. JAIR 9: 99–137Google Scholar
  4. 4.
    Blaylock, N., Allen, J., & Ferguson, G. (2003). Managing communicative intentions with collaborative problem solving. In Current and new directions in dialogue. Kluwer.Google Scholar
  5. 5.
    Boutilier C., Brafman R. (2001) Partial order planning with concurrent interacting actions. JAIR 14: 105–136zbMATHGoogle Scholar
  6. 6.
    Brenner, M. (2005). Planning for multiagent environments: From individual perceptions to coordinated execution. In Wsh. Multiagent Planning and Scheduling, ICAPS ’05, Monterey, USA.Google Scholar
  7. 7.
    Brenner, M. (2007). Situation-aware interpretation, planning and execution of user commands by autonomous robots. In Proceedings of IEEE RO-MAN 2007.Google Scholar
  8. 8.
    Brenner, M., Hawes, N. A., Kelleher, J. D., & Wyatt, J. (2007). Mediating between qualitative and quantitative representations for task-orientated human–robot interaction. In Proceedings of IJCAI 2007.Google Scholar
  9. 9.
    Brenner, M., & Kruijff-Korbayová, I. (2008). A continual multiagent planning approach to situated dialogue. In Proceedings of the 12th workshop on the semantics and pragmatics of dialogue (semdial), London, UK.Google Scholar
  10. 10.
    Clark, H. H., & Marshall, C. R. (1981). Definite reference and mutual knowledge. In Elements of discourse understanding. Cambridge University Press.Google Scholar
  11. 11.
    Clement, B., & Barrett, A. (2003). Continual coordination through shared activities. In Proceedings of AAMAS ’03.Google Scholar
  12. 12.
    Clement, B., & Durfee, E. (1999). Top-down search for coordinating the hierarchical plans of multiple agents. In Proceedings of AGENTS ’99.Google Scholar
  13. 13.
    Cox, J. S., & Durfee, E. H. (2005). An efficient algorithm for multiagent plan coordination. In Proceedings of AAMAS ’05.Google Scholar
  14. 14.
    DesJardins M., Durfee E., Ortiz C. Jr., Wolverton M. (1999) A survey of research in distributed, continual planning. The AI Magazine 20(4): 13–22Google Scholar
  15. 15.
    DesJardins, M., & Wolverton, M. (1999). Coordinating a distributed planning system. The AI Magazine, 20(4).Google Scholar
  16. 16.
    Durfee E., Montgomery T. (1991) Coordination as distributed search in hierarchical behavior space. IEEE Transactions on Systems, Man, and Cybernetics 21(6): 1363–1378CrossRefGoogle Scholar
  17. 17.
    Durfee E.H. (1999) Distributed continual planning for unmanned ground vehicle teams. AI Magazine 20(4): 55–61Google Scholar
  18. 18.
    Edelkamp, S., & Hoffmann, J. (2004). PDDL2.2: The language for the classical part of the 4th international planning competition. Technical Report 195, Albert-Ludwigs-Universität, Institut für Informatik, Freiburg, Germany.Google Scholar
  19. 19.
    Erol K., Hendler J., Nau D. (1996) Complexity results for hierarchical task-network planning. Annals of Mathematics and Artificial Intelligence 18: 69–93zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Etzioni O., Golden K., Weld D.S. (1997) Sound and efficient closed-world reasoning for planning. Artificial Intelligence 89(1–2): 113–148zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Etzioni, O., Hanks, S., Weld, D., Draper, D., Lesh, N., & Williamson, M. (1992). An approach to planning with incomplete information. In Proceedings of KR-92 (pp. 115–125).Google Scholar
  22. 22.
    Fagin, R., Halpern, J. Y., Moses, Y., & Vardi, M. Y. (1995). Reasoning about knowledge. MIT Press.Google Scholar
  23. 23.
    Fox M., Long D. (2003) PDDL 2.1: An extension to PDDL for expressing temporal planning domains. JAIR 20: 61–124zbMATHGoogle Scholar
  24. 24.
    Geffner H. (2000) Functional STRIPS: A more flexible language for planning and problem solving. In: Minker J. (eds) Logic-based artificial intelligence. Kluwer, Dordrecht, HollandGoogle Scholar
  25. 25.
    Golden, K. (1998). Leap before you look: Information gathering in the puccini planner. In Proceedings of AIPS-98 (pp. 70–77).Google Scholar
  26. 26.
    Golden, K., & Weld, D. (1996). Representing sensing actions: The middle ground revisited. In Proceedings of KR ’96.Google Scholar
  27. 27.
    Grosz, B. J., & Kraus, S. (1996). Collaborative plans for complex group action. Artificial Intelligence, 86.Google Scholar
  28. 28.
    Helmert M. (2006) The Fast Downward planning system. Journal of Artificial Intelligence Research 26: 191–246zbMATHCrossRefGoogle Scholar
  29. 29.
    Hobbs, J.R., Moore, R.C. (eds) (1985) Formal theories of the commonsense world. Ablex, Norwood, NJGoogle Scholar
  30. 30.
    Hoffmann, J., & Brafman, R. (2005). Contingent planning via heuristic forward search with implicit belief states. In S. Biundo, K. L. Myers, & K. Rajan (Eds.), ICAPS (pp. 71–80). AAAI.Google Scholar
  31. 31.
    Hoffmann J., Brafman R.I. (2006) Conformant planning via heuristic forward search: A new approach. Artificial Intelligence 170: 507–541zbMATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Kambhampati, S. (2007). Model-lite planning for the web age masses: The challenges of planning with incomplete and evolving domain models. In Proceedings of AAAI-07.Google Scholar
  33. 33.
    Knoblock C.A. (1995) Planning, executing, sensing, and replanning for information gathering. In: Mellish C. (eds) Proceedings of the fourteenth international joint conference on artificial intelligence. Morgan Kaufmann, San Francisco, pp 1686–1693Google Scholar
  34. 34.
    Kruijff, G.-J., & Brenner, M. (2007). Modelling spatio-temporal comprehension in situated human–robot dialogue as reasoning about intentions and plans. In AAAI spring symposium on intentions in intelligent systems, Stanford, CA.Google Scholar
  35. 35.
    Levesque, H. J. (1996). What is planning in the presence of sensing? In Proceedings of AAAI-96 (pp. 1139–1146). MIT Press.Google Scholar
  36. 36.
    Lewis D. (1969) Convention. A philosophical study. Harvard University Press, Cambridge, MAGoogle Scholar
  37. 37.
    Littman M., Goldsmith J., Mundhenk M. (1998) The computational complexity of probabilistic planning. JAIR 9: 1–36zbMATHMathSciNetGoogle Scholar
  38. 38.
    Lochbaum K.E. (1998) A collaborative planning model of intentional structure. Computational Linguistics 24: 525–572Google Scholar
  39. 39.
    Lynch N. (1996) Distributed algorithms. Morgan Kaufmann, San Francisco, CAzbMATHGoogle Scholar
  40. 40.
    McDermott, D. (1998). PDDL—the planning domain definition language. Technical Report TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control.Google Scholar
  41. 41.
    Myers, K. L. (1999). Cpef: A continuous planning and execution framework. The AI Magazine, 20(4).Google Scholar
  42. 42.
    Nau D., Cao Y., Lotem A., Munoz-Avila H. (1999) SHOP: Simple hierarchical ordered planner. In: Dean T. (eds) Proceedings of the 16th international joint conference on artificial intelligence (IJCAI-99). Morgan Kaufmann, Stockholm, SwedenGoogle Scholar
  43. 43.
    Petrick, R., & Bacchus, F. (2002). A knowledge-based approach to planning with incomplete information and sensing. In Proceedings of AIPS-02.Google Scholar
  44. 44.
    Petrick, R. P. A., & Bacchus, F. (2004). Extending the knowledge-based approach to planning with incomplete information and sensing. In Proceedings of ICAPS 2004 (pp. 2–11).Google Scholar
  45. 45.
    Ram, A., Leake, D.B. (eds) (1995) Goal-driven learning. MIT Press, Cambridge, MAzbMATHGoogle Scholar
  46. 46.
    Rich C., Sidner C.L. (1998) Collagen: A collaboration manager for software interface agents. UMUAI 8(3–4): 315–350Google Scholar
  47. 47.
    Rintanen J. (1999) Constructing conditional plans by a theorem-prover. JAIR 10: 323–352zbMATHGoogle Scholar
  48. 48.
    Russell S., Norvig P. (2003) Artificial intelligence: A modern approach, 2nd edn. Prentice-Hall, Englewood Cliffs, NJGoogle Scholar
  49. 49.
    Sadek, M. D. (1991). Dialogue acts are rational plans. In Proceedings of the ESCA/ETR workshop on multi-modal dialogue. Italy: Maratea.Google Scholar
  50. 50.
    Smith, D. E., & Weld, D. S. (1998). Conformant graphplan. In AAAI/IAAI (pp. 889–896).Google Scholar
  51. 51.
    Thiebaux, S., Hoffmann, J., & Nebel, B. (2003). In defense of axioms in PDDL. In Proceedings of IJCAI.Google Scholar
  52. 52.
    Traum, D. (1994). A computational model of grounding in natural language conversation. PhD thesis, University of Rochester.Google Scholar
  53. 53.
    Weld, D. S., Anderson, C. R., & Smith, D. E. (1998). Extending graphplan to handle uncertainty and sensing actions. In AAAI/IAAI (pp. 897–904).Google Scholar
  54. 54.
    Yang, Q. (1997). Intelligent planning: A decomposition and abstraction based approach. Springer.Google Scholar
  55. 55.
    Yokoo, M., & Hirayama, K. (2000). Algorithms for distributed constraint satisfaction: A review. Autonomous Agents and Multi-Agent Systems, 3(2).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Institute for Computer ScienceAlbert-Ludwigs-UniversitätFreiburgGermany

Personalised recommendations