Skip to main content
SpringerLink
Log in

Multi-agent Simulation for AI Behaviour Discovery in Operations Research

  • Conference paper
  • First Online:
Multi-Agent-Based Simulation XXII (MABS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13128))

  • 525 Accesses

Abstract

We describe ACE0, a lightweight platform for evaluating the suitability and viability of AI methods for behaviour discovery in multi-agent simulations. Specifically, ACE0 was designed to explore AI methods for multi-agent simulations used in operations research studies related to new technologies such as autonomous aircraft. Simulation environments used in production are often high-fidelity, complex, require significant domain knowledge and as a result have high R&D costs. Minimal and lightweight simulation environments can help researchers and engineers evaluate the viability of new AI technologies for behaviour discovery in a more agile and potentially cost effective manner. In this paper we describe the motivation for the development of ACE0. We provide a technical overview of the system architecture, describe a case study of behaviour discovery in the aerospace domain, and provide a qualitative evaluation of the system. The evaluation includes a brief description of collaborative research projects with academic partners, exploring different AI behaviour discovery methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This is a simplified view of the situation, as one can consider higher order features such as turn rates and other time derivatives of the basic state space variables.

References

  1. Austin, F., Carbone, G., Lewis, M.: Automated maneuvering decisions for air-to-air combat. In: In Proceedings of the Military Computing Conference. Anaheim, California, May 1987

    Google Scholar 

  2. Burgin, G.H., Sidor, L.B.: Rule-Based Air Combat Simulation. Technical Report, Contractor Report 4160, National Aeronautics and Space Administration (NASA) (1988)

    Google Scholar 

  3. Colledanchise, M., Ögren, P.: How behavior trees modularize hybrid control systems and generalize sequential behavior compositions, the subsumption architecture, and decision trees. IEEE Trans. Rob. 33(2), 372–389 (2017). https://doi.org/10.1109/TRO.2016.2633567

    Article  Google Scholar 

  4. Evertsz, R., Thangarajah, J., Papasimeon, M.: The conceptual modelling of dynamic teams for autonomous systems. In: Mayr, H.C., Guizzardi, G., Ma, H., Pastor, O. (eds.) ER 2017. LNCS, vol. 10650, pp. 311–324. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69904-2_25

    Chapter  Google Scholar 

  5. Heinze, C., Papasimeon, M., Goss, S., Cross, M., Connell, R.: Simulating fighter pilots. In: Pěchouček, M., Thompson, S.G., Voos, H. (eds.) Defence Industry Applications of Autonomous Agents and Multi-Agent Systems. WSSAT, pp. 113–130. Birkhäuser Basel, Basel (2008). http://dx.doi.org/10.1007/978-3-7643-8571-2_7

  6. Hossam, M., Le, T., Huynh, V., Papasimeon, M., Phung, D.Q.: OptiGAN: generative adversarial networks for goal optimized sequence generation. In: International Joint Conference on Neural Networks (IJCNN). Glasgow, Scotland, UK, July 2020

    Google Scholar 

  7. Hossam, M., Le, T., Papasimeon, M., Huynh, V., Phung, D.: Text generation with deep variational GAN. In: NeurIPS 3rd Workshop on Bayesian Deep Learning. Montreal, Canada, December 2018. http://bayesiandeeplearning.org/2018/papers/157.pdf

  8. Jones, R.M., Wray, R., van Lent, M., Laird, J.E.: Planning in the Tactical Air Domain. Technical Report, aAAI Technical Report FS-94-01, AAAI (1994)

    Google Scholar 

  9. Kluyver, T., et al.: Jupyter Notebooks - a publishing format for reproducible computational workflows. In: Positioning and Power in Academic Publishing: Players, Agents and Agendas, pp. 87–90. IOS Press (2016). https://doi.org/10.3233/978-1-61499-649-1-87

  10. Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: An empirical study of reward structures for actor-critic reinforcement learning in air combat manoeuvring simulation. In: Liu, J., Bailey, J. (eds.) AI 2019. LNCS (LNAI), vol. 11919, pp. 54–65. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35288-2_5

    Chapter  Google Scholar 

  11. Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: Discrete-to-Deep supervised policy learning: an effective training method for neural reinforcement learning. In: ALA 2020: Adaptive Learning Agents Workshop at AAMAS 2020. Auckland, New Zealand (2020)

    Google Scholar 

  12. Lam, C.P., Masek, M., Kelly, L., Papasimeon, M., Benke, L.: A simheuristic approach for evolving agent behaviour in the exploration for novel combat tactics. Oper. Res. Perspect. 6, 100123 (2019). https://doi.org/10.1016/j.orp.2019.100123

    Article  MathSciNet  Google Scholar 

  13. Lipovetzky, N., Geffner, H.: Width and serialization of classical planning problems. In: ECAI, pp. 540–545 (2012)

    Google Scholar 

  14. Martzinotto, A., Colledanchise, M., Smith, C., Ögren, P.: Towards a unified behavior trees framework for robot control. In: In proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), pp. 5420–5427 (2014). https://doi.org/10.1109/icra.2014.6907656

  15. Masek, M., Lam, C.P., Benke, L., Kelly, L., Papasimeon, M.: Discovering emergent agent behaviour with evolutionary finite state machines. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds.) PRIMA 2018. LNCS (LNAI), vol. 11224, pp. 19–34. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03098-8_2

    Chapter  Google Scholar 

  16. Masek, M., Lam, C.P., Kelly, L., Benke, L., Papasimeon, M.: A genetic programming framework for novel behaviour discovery in air combat scenarios. In: Ernst, A.T., Dunstall, S., García-Flores, R., Grobler, M., Marlow, D. (eds.) Data and Decision Sciences in Action 2. LNMIE, pp. 263–277. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60135-5_19

    Chapter  Google Scholar 

  17. McGrew, J.S., How, J.P.: Air combat strategy using approximate dynamic programming. J. Guidance Control Dyn. 33, 1641–1654 (2010)

    Article  Google Scholar 

  18. Papasimeon, M., Pearce, A., Goss, S.: The human agent virtual environment. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. AAMAS 2007. Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1329125.1329463

  19. Park, H., Lee, B.Y., Tahk, M.J., Yoo, D.W.: Differential game based air combat maneuver generation using scoring function matrix. Int. J. Aeronaut. Space Sci. 17(2), 204–213 (2016)

    Article  Google Scholar 

  20. Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9(3), 21–29 (2007)

    Article  Google Scholar 

  21. Ramirez, M., Papasimeon, M., Benke, L., Lipovetzky, N., Miller, T., Pearce, A.R.: Real-Time UAV maneuvering via automated planning in simulations. In: 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 5243–5245. Melbourne, Australia, August 2017

    Google Scholar 

  22. Ramirez, M., et al.: Integrated hybrid planning and programmed control for real time UAV maneuvering. In: 17th Int. Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 1318–1326. Stockholm, Sweden, July 2018

    Google Scholar 

  23. Shaw, R.L.: Fighter Combat: Tactics and Maneuvering. Naval Institute Press, Annapolis (1985)

    Google Scholar 

  24. Smith, R., Dike, B., Ravichandran, B., El-Fallah, A., Mehra, K.: Discovering novel fighter combat maneuvers: simulating test pilot creativity. In: Bentley, P.J., Corne, D.W. (eds.) Creative Evolutionary Systems, p. 467 - VIII. The Morgan Kaufmann Series in Artificial Intelligence, Morgan Kaufmann, San Francisco (2002). https://doi.org/10.1016/B978-155860673-9/50059-8

  25. Tidhar, G., Heinze, C., Selvestrel, M.: Flying together: modelling air mission teams. Appl. Intell. 8(3), 195–218 (1998)

    Article  Google Scholar 

  26. Toubman, A.: Calculated Moves: Generating Air Combat Behaviour. Ph.D. thesis, Leiden University, The Netherlands (2020)

    Google Scholar 

  27. Vered, M., Kaminka, G.A.: Heuristic online goal recognition in continuous domains. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17), pp. 4447–4454. Melbourne, Australia (2017)

    Google Scholar 

  28. Zhang, L.A., et al.: Air Dominance Through Machine Learning: A Preliminary Exploration of Artificial Intelligence-Assisted Mission Planning. Technical Report, RR4311, RAND Corporation, Santa Monica, CA, USA (2020). https://www.rand.org/pubs/research_reports/RR4311.html

Download references

Author information

Authors and Affiliations

  1. Defence Science and Technology Group, 506 Lorimer Street, Fishermans Bend, VIC, 3207, Australia

    Michael Papasimeon & Lyndon Benke

Authors
  1. Michael Papasimeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lyndon Benke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Papasimeon .

Editor information

Editors and Affiliations

  1. Imperial College London, London, UK

    Koen H. Van Dam

  2. Toulouse 1 Capitole University, Toulouse, France

    Nicolas Verstaevel

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Papasimeon, M., Benke, L. (2022). Multi-agent Simulation for AI Behaviour Discovery in Operations Research. In: Van Dam, K.H., Verstaevel, N. (eds) Multi-Agent-Based Simulation XXII. MABS 2021. Lecture Notes in Computer Science(), vol 13128. Springer, Cham. https://doi.org/10.1007/978-3-030-94548-0_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-94548-0_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-94547-3

  • Online ISBN: 978-3-030-94548-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation