Autonomous Agents and Multi-Agent Systems

, Volume 32, Issue 5, pp 602–634 | Cite as

Landmark-based heuristic online contingent planning

  • Shlomi Maliah
  • Guy ShaniEmail author
  • Ronen I. Brafman


In contingent planning problems, agents have partial information about their state and use sensing actions to learn the value of some variables. When sensing and actuation are separated, plans for such problems can often be viewed as a tree of sensing actions, separated by conformant plans consisting of non-sensing actions that enable the execution of the next sensing action. We propose a heuristic, online method for contingent planning which focuses on identifying the next useful sensing action. We select the next sensing action based on a landmark heuristic, adapted from classical planning. We discuss landmarks for plan trees, providing several alternative definitions and discussing their merits. The key part of our planner is the novel landmarks-based heuristic, together with a projection method that uses classical planning to solve the intermediate conformant planning problems. The resulting heuristic contingent planner solves many more problems than state-of-the-art, translation-based online contingent planners, and in most cases, much faster, up to 3 times faster on simple problems, and 200 times faster on non-simple domains.


Contingent planning Partial observability Online planning Landmarks Automated planning Regression Belief space 



We thank the reviewers for their useful comments. This work was supported by ISF Grant 933/13, by the Israel Ministry of Science and Technology Grant 54178, by the Helmsley Charitable Trust through the Agricultural, Biological and Cognitive Robotics Center of Ben-Gurion University of the Negev, and by the Lynn and William Frankel Center for Computer Science.


  1. 1.
    Albore, A., Palacios, H., & Geffner, H. (2009). A translation-based approach to contingent planning. In: IJCAI, pp. 1623–1628.Google Scholar
  2. 2.
    Bonet, B., & Geffner, H. (2000). Planning with incomplete information as heuristic search in belief space. In: Proceedings of AIPS’00, pp. 52–61.Google Scholar
  3. 3.
    Bonet, B., & Geffner, H. (2011). Planning under partial observability by classical replanning: Theory and experiments. In: IJCAI, pp. 1936–1941.Google Scholar
  4. 4.
    Brafman, R. I., & Shani, G. (2012). A multi-path compilation approach to contingent planning. In: AAAI.Google Scholar
  5. 5.
    Brafman, R. I., & Shani, G. (2012). Replanning in domains with partial information and sensing actions. Journal of Artificial Intelligence Research (JAIR), 45, 565–600.MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Brafman, R. I., & Shani, G. (2016). Online belief tracking using regression for contingent planning. Artificial Intelligence, 241, 131–152.MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Bryce, D., Kambhampati, S., & Smith, D. E. (2006). Planning graph heuristics for belief space search. Journal of AI Research, 26, 35–99.zbMATHGoogle Scholar
  8. 8.
    Heckerman, D., Horvitz, E., & Middleton, B. (1993). An approximate nonmyopic computation for value of information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(3), 292–298.CrossRefGoogle Scholar
  9. 9.
    Helmert, M. (2006). The fast downward planning system. Journal of Artificial Intelligence Research, 26, 191–246.CrossRefzbMATHGoogle Scholar
  10. 10.
    Hoffmann, J., & Nebel, B. (2001). The FF planning system: Fast plan generation through heuristic search. JAIR, 14, 253–302.CrossRefzbMATHGoogle Scholar
  11. 11.
    Hoffmann, J., Porteous, J., & Sebastia, L. (2004). Ordered landmarks in planning. Journal of AI Research, 22, 215–278.MathSciNetzbMATHGoogle Scholar
  12. 12.
    Howard, R. A. (1960). Dynamic programming and Markov processes. Cambridge: MIT Press.zbMATHGoogle Scholar
  13. 13.
    Karpas, E., & Domshlak, C. (2009). Cost-optimal planning with landmarks. In: IJCAI.Google Scholar
  14. 14.
    Keyder, E., Richter, S., & Helmert, M. (2010). Sound and complete landmarks for and/or graphs. In: Proceedings of the European Conference on Artificial Intelligence (ECAI), pp. 335–340.Google Scholar
  15. 15.
    Komarnitsky, R., & Shani, G. (2016). Computing contingent plans using online replanning. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12–17, 2016, Phoenix, Arizona, USA, pp. 3159–3165.Google Scholar
  16. 16.
    Maliah, S., Brafman, R. I., Karpas, E., & Shani, G. (2014). Partially observable online contingent planning using landmark heuristics. In: Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling, ICAPS 2014, Portsmouth, New Hampshire, USA, June 21–26, 2014.Google Scholar
  17. 17.
    Maliah, S., Shani, G., & Brafman, R. I. (2016). Online macro generation for privacy preserving planning. In: Proceedings of the Twenty-Sixth International Conference on Automated Planning and Scheduling, ICAPS 2016, London, UK, June 12–17, 2016, pp. 216–220.Google Scholar
  18. 18.
    Newton, M. A. H. (2009). Wizard: Learning macro-actions comprehensively for planning. Ph.D. thesis, University of Strathclyde.Google Scholar
  19. 19.
    Richter, S., & Westphal, M. (2010). The LAMA planner: Guiding cost-based anytime planning with landmarks. JAIR, 39, 127–177.CrossRefzbMATHGoogle Scholar
  20. 20.
    Richter, S., & Westphal, M. (2010). The lama planner: Guiding cost-based anytime planning with landmarks. Journal of Artificial Intelligence Research, 39(1), 127–177.CrossRefzbMATHGoogle Scholar
  21. 21.
    Rintanen, J., Heljanko, K., & Niemelä, I. (2006). Planning as satisfiability: Parallel plans and algorithms for plan search. Artificial Intelligence, 170(12–13), 1031–1080.MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Shani, G., Brafman, R. I. (2011). Replanning in domains with partial information and sensing actions. In: IJCAI, pp. 2021–2026.Google Scholar
  23. 23.
    Smith, T., & Simmons, R. (2004). Heuristic search value iteration for POMDPs. In: UAI 2004. Banff, Alberta.Google Scholar
  24. 24.
    Speck, D., Ortlieb, M., & Mattmüller, R. (2015). Necessary observations in nondeterministic planning. In: KI 2015: Advances in Artificial Intelligence—38th Annual German Conference on AI, Dresden, Germany, September 21–25, 2015, Proceedings, pp. 181–193.Google Scholar
  25. 25.
    To, S. T., Son, T. C., & Pontelli, E. (2015). A generic approach to planning in the presence of incomplete information: Theory and implementation. Artificial Intelligence, 227, 1–51.MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    To, S. T., Pontelli, E., & Son, T. C. (2009). A conformant planner with explicit disjunctive representation of belief states. In: ICAPS.Google Scholar
  27. 27.
    Zhu, L., & Givan, R. (2003). Landmark extraction via planning graph propagation. In: ICAPS 2003 Doctoral Consortium, pp. 156–160.Google Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Information Systems EngineeringBen Gurion University of the NegevBeer-ShevaIsrael
  2. 2.Computer Science DepartmentBen Gurion University of the NegevBeer-ShevaIsrael

Personalised recommendations