Skip to main content
SpringerLink
Log in

On the Importance of Domain Model Configuration for Automated Planning Engines

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

The development of domain-independent planners within the AI planning community is leading to “off-the-shelf” technology that can be used in a wide range of applications. Moreover, it allows a modular approach—in which planners and domain knowledge are modules of larger software applications—that facilitates substitutions or improvements of individual modules without changing the rest of the system. This approach also supports the use of reformulation and configuration techniques, which transform how a model is represented in order to improve the efficiency of plan generation. In this article, we investigate how the performance of domain-independent planners is affected by domain model configuration, i.e. the order in which elements are ordered in the model, particularly in the light of planner comparisons. We then introduce techniques for the online and offline configuration of domain models, and we analyse the impact of domain model configuration on other reformulation approaches, such as macros.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://ipc.icaps-conference.org.

  2. We found optimising for median performance to not be a good idea, since it can yield configurations that work very well on 51% of the instances but extremely poorly on others; in contrast, the mean is often quite dominated by the worst cases, and optimising it therefore also reduces the failure rate. Our performance metric m can also already penalise failures substantially, allowing us to use Eq. 1 to truly minimise the failure rate and only break ties by the average performance in non-failure cases.

  3. This is due to their grounding, as macros tend to have many parameters derived from the encapsulated operators, and add to the increased branching factor of the search space.

References

  1. Ai-Chang, M., Bresina, J., Charest, L., Chase, A., Hsu, J.C.-J., Jonsson, A., Kanefsky, B., Morris, P., Rajan, K., Yglesias, J., et al.: Mapgen: mixed-initiative planning and scheduling for the mars exploration rover mission. IEEE Intell. Syst. 19(1), 8–12 (2004)

    Article  Google Scholar 

  2. Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: Proceedings of the Principles and Practice of Constraint Programming (CP), pp. 142–157 (2009)

  3. Areces, C., Bustos, F., Dominguez, M.A., Hoffmann, J.: Optimizing planning domains by automatic action schema splitting. In: Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling, ICAPS (2014)

  4. Balyo, T.: The freelunch planning system entering IPC 2014. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  5. Balyo, T.: Relaxing the relaxed exist-step parallel planning semantics. In: 2013 IEEE 25th International Conference on Tools with Artificial Intelligence, pp. 865–871 (2013)

  6. Botea, A., Enzenberger, M., Müller, M., Schaeffer, J.: Macro-FF: Improving AI planning with automatically learned macro-operators. J. Artif. Intell. Res. 24, 581–621 (2005)

    Article  Google Scholar 

  7. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  8. Cenamor, I., de la Rosa, T., Fernández, F.: The ibacop planning system: instance-based configured portfolios. J. Artif. Intell. Res. 56, 657–691 (2016)

    Article  MathSciNet  Google Scholar 

  9. Cerutti, F., Vallati, M., Giacomin, M.: On the impact of configuration on abstract argumentation automated reasoning. Int. J. Approx. Reason. 92, 120 (2017)

    Article  MathSciNet  Google Scholar 

  10. Chrpa, L.: Generation of macro-operators via investigation of action dependencies in plans. Knowl. Eng. Rev. 25(3), 281–297 (2010)

    Article  Google Scholar 

  11. Chrpa, L., Barták, R.: Reformulating planning problems by eliminating unpromising actions. In: Symposium on Abstraction, Reformulation, and Approximation, SARA, pp. 50–57 (2009)

  12. Chrpa, L., McCluskey, T.L.: On exploiting structures of classical planning problems: generalizing entanglements. In: European Conference on Artificial Intelligence, ECAI, pp. 240–245 (2012)

  13. Chrpa, L., Vallati, M., McCluskey, T.L.: Mum: A technique for maximising the utility of macro-operators by constrained generation and use. In: Proceedings of the International Conference on Automated Planning and Scheduling, ICAPS, pp. 65–73 (2014)

  14. Coles, A., Fox, M., Smith, A.: Online identification of useful macro-actions for planning. In: The International Conference on Automated Planning and Scheduling, ICAPS, pp. 97–104 (2007)

  15. Fawcett, C., Helmert, M., Hoos, H.H., Karpas, E., Röger, G., Seipp, J.: Fd-autotune: domain-specific configuration using fast-downward. In: Workshop on Planning and Learning (PAL) (2011)

  16. Fox, M., Long, D.: PDDL2.1: an extension to PDDL for expressing temporal planning domains. J. Artif. Intell. Res. 20, 61–124 (2003)

    Article  Google Scholar 

  17. Gerevini, A.E., Saetti, A., Serina, I.: Planning through stochastic local search and temporal action graphs in LPG. J. Artif. Intell. Res. 20, 239–290 (2003)

    Article  Google Scholar 

  18. Gerevini, A.E., Saetti, A., Vallati, M.: Planning through automatic portfolio configuration: the PBP approach. J. Artif. Intell. Res. 50, 639–696 (2014)

    Article  Google Scholar 

  19. Ghallab, M., Knoblock Isi, C., Penberthy, S., Smith, D.E., Sun, Y., Weld, D.: Pddl—the planning domain definition language. Technical report (1998)

  20. Ghallab, M., Nau, D., Traverso, P.: Automated Planning, Theory and Practice. Morgan Kaufmann Publishers, Burlington (2004)

    MATH  Google Scholar 

  21. Helmert, M.: The fast downward planning system. J. Artif. Intell. Res. 26, 191–246 (2006)

    Article  Google Scholar 

  22. Howe, A.E., Dahlman, E.: A critical assessment of benchmark comparison in planning. J. Artif. Intell. Res. 17, 1–33 (2002)

    Article  Google Scholar 

  23. Hsu, C.-W., Wah, B.W.: The SGPlan planning system in IPC-6. In: The 6th International Planning Competition (IPC-6) (2008)

  24. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Proceedings of the 5th Learning and Intelligent Optimization Conference (LION), pp. 507–523 (2011)

  25. Hutter, F., Hoos, H.H., Leyton-Brown, K.: An efficient approach for assessing hyperparameter importance. In: Proceedings of the 31st International Conference on Machine Learning, pp. 754–762 (2014)

  26. Hutter, F., Hoos, H.H., Leyton-Brown, K., Stützle, T.: Paramils: an automatic algorithm configuration framework. J. Artif. Intell. Res. 36, 267–306 (2009)

    Article  Google Scholar 

  27. Hutter, F., Lindauer, M., Balint, A., Bayless, S., Hoos, H., Leyton-Brown, K.: The configurable sat solver challenge (CSSC). Artif. Intell. J. 243, 1–25 (2017)

    Article  MathSciNet  Google Scholar 

  28. Hutter, F., Xu, L., Hoos, H.H., Leyton-Brown, K.: Algorithm runtime prediction: methods & evaluation. Artif. Intell. 206, 79–111 (2014)

    Article  MathSciNet  Google Scholar 

  29. Katz, M., Hoffmann, J.: Mercury planner: pushing the limits of partial delete relaxation. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  30. Korf, R.E.: Macro-operators: a weak method for learning. Artif. Intell. 26(1), 35–77 (1985)

    Article  MathSciNet  Google Scholar 

  31. Lipovetzky, N., Ramirez, M., Muise, C., Geffner, H.: Width and inference based planners: Siw, bfs(f), and probe. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  32. McCluskey, T.L., Porteous, J.M.: Engineering and compiling planning domain models to promote validity and efficiency. Artif. Intell. 95(1), 1–65 (1997)

    Article  Google Scholar 

  33. McCluskey, T.L., Vallati, M.: Embedding automated planning within urban traffic management operations. In: Proceedings of the International Conference on Automated Planning and Scheduling ICAPS (2017)

  34. McCluskey, T.L., Vaquero, T.S., Vallati, M.: Engineering knowledge for automated planning: towards a notion of quality. In: Proceedings of the Knowledge Capture Conference, K-CAP (2017)

  35. Minton, S.: Quantitative results concerning the utility of explanation-based learning. In: AAAI, pp. 564–569 (1988)

  36. Newton, M.A.H., Levine, J., Fox, M., Long, D.: Learning macro-actions for arbitrary planners and domains. In: The International Conference on Automated Planning and Scheduling, ICAPS, pp. 256–263 (2007)

  37. Parkinson, S., Longstaff, A.P.: Multi-objective optimisation of machine tool error mapping using automated planning. Expert Syst. Appl. 42(6), 3005–3015 (2015)

    Article  Google Scholar 

  38. Riddle, P.J., Holte, R.C., Barley, M.W.: Does representation matter in the planning competition? In: Proceedings of the Ninth Symposium on Abstraction, Reformulation, and Approximation, SARA 2011, Parador de Cardona, Cardona, Catalonia, Spain (2011)

  39. Rintanen, J.: Madagascar: Scalable planning with SAT. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  40. Sadraei, R., Ahmadi, A.: Use: the useful operator selection. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  41. Seipp, J., Sievers, S., Helmert, M., Hutter, F.: Automatic configuration of sequential planning portfolios. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 25–30 Jan 2015, Austin, Texas, USA, pp. 3364–3370 (2015)

  42. Thompson, S.K.: Simple random sampling. Sampling, Third Edition, pp. 9–37 (2012)

  43. Valenzano, R., Nakhost, H., Müller, M., Schaeffer, J.: Arvandherd 2014. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  44. Valenzano, R., Schaeffer, J., Sturtevant, N., Xie, F.: A comparison of knowledge-based GBFS enhancements and knowledge-free exploration. In: Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS-2014), pp. 375–379 (2014)

  45. Vallati, M., Fawcett, C., Gerevini, A.E., Hoos, H.H., Saetti, A.: Automatic generation of efficient domain-optimized planners from generic parametrized planners. In: Proceedings of the Sixth Annual Symposium on Combinatorial Search, SOCS (2013)

  46. Vallati, M., Chrpa, L., Grzes, M., McCluskey, T.L., Roberts, M.: The 2014 international planning competition: progress and trends. AI Mag. 36(3), 90–98 (2015)

    Google Scholar 

  47. Vallati, M., Chrpa, L., McCluskey, T.L.: Improving a planner’s performance through online heuristic configuration of domain models. In: Proceedings of the Tenth International Symposium on Combinatorial Search, pp. 171–172 (2017)

  48. Vallati, M., Hutter, F., Chrpa, L., McCluskey, T.L.: On the effective configuration of planning domain models. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI, pp. 1704–1711 (2015)

  49. Vallati, M., Maratea, M.: On the configuration of SAT formulae. In: AI*IA 2019—Advances in Artificial Intelligence—XVIIIth International Conference of the Italian Association for Artificial Intelligence, pp. 264–277 (2019)

  50. Vidal, V.: YAHSP3 and YAHSP3-MT in the 8th international planning competition. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  51. Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945)

    Article  Google Scholar 

  52. Xie, F., Müller, M., Holte, R.: Jasper: the art of exploration in greedy best first search. In: Proceedings of the 8th International Planning Competition (IPC-2014) (2014)

  53. Yuan, Z., Stützle, T., Birattari, M.: Mads/f-race: mesh adaptive direct search meets f-race. In: Proceedings of the 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE), pp. 41–50 (2010)

Download references

Acknowledgements

The authors would like to acknowledge the use of the University of Huddersfield Queensgate Grid in carrying out this work. This Research was partially funded by the Czech Science Foundation (Project No. 18-07252S).

Author information

Authors and Affiliations

  1. School of Computing and Engineering, University of Huddersfield, Huddersfield, UK

    Mauro Vallati & Thomas Leo McCluskey

  2. Artificial Intelligence Center, Czech Technical University in Prague, Prague, Czech Republic

    Lukáš Chrpa

  3. Faculty of Mathematics and Physics, Charles University in Prague, Prague, Czech Republic

    Lukáš Chrpa

  4. University of Freiburg, Freiburg, Germany

    Frank Hutter

Authors
  1. Mauro Vallati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lukáš Chrpa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Leo McCluskey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Frank Hutter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mauro Vallati.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Domain-by-domain results

Domain-by-domain results

See Tables 14, 15, 16, 17 and 18.

Table 14 Results show the best (B), worst (W), median (M) PAR10 and coverage performance achieved on the IPC 2014 benchmarks, running planners on 50 randomly configured domain models
Full size table
Table 15 Results show the best (B), worst (W), median (M) PAR10 and coverage performance achieved on the IPC 2014 benchmarks, running planners on 50 randomly configured domain models
Full size table
Table 16 Results show the best (B), worst (W), median (M) PAR10 and coverage performance achieved on the IPC 2014 benchmarks, running planners on 50 randomly configured domain models
Full size table
Table 17 Results show the best (B), worst (W), median (M) PAR10 and coverage performance achieved on the IPC 2014 benchmarks, running planners on 50 randomly configured domain models
Full size table
Table 18 Results show the best (B), worst (W), median (M) PAR10 and coverage performance achieved on the IPC 2014 benchmarks, running planners on 50 randomly configured domain models
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vallati, M., Chrpa, L., McCluskey, T.L. et al. On the Importance of Domain Model Configuration for Automated Planning Engines. J Autom Reasoning 65, 727–773 (2021). https://doi.org/10.1007/s10817-021-09592-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10817-021-09592-1

Keywords

Search

Navigation