Skip to main content
SpringerLink
Log in

Game-theoretic multi-agent motion planning in a mixed environment

  • Research Article
  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

The motion planning problem for multi-agent systems becomes particularly challenging when humans or human-controlled robots are present in a mixed environment. To address this challenge, this paper presents an interaction-aware motion planning approach based on game theory in a receding-horizon manner. Leveraging the framework provided by dynamic potential games for handling the interactions among agents, this approach formulates the multi-agent motion planning problem as a differential potential game, highlighting the effectiveness of constrained potential games in facilitating interactive motion planning among agents. Furthermore, online learning techniques are incorporated to dynamically learn the unknown preferences and models of humans or human-controlled robots through the analysis of observed data. To evaluate the effectiveness of the proposed approach, numerical simulations are conducted, demonstrating its capability to generate interactive trajectories for all agents, including humans and human-controlled agents, operating within the mixed environment. The simulation results illustrate the effectiveness of the proposed approach in handling the complexities of multi-agent motion planning in real-world scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Yarlagadda, R. T. (2015). Future of robots, AI and automation in the United States. IEJRD-International Multidisciplinary Journal, 1(5), 6.

    Google Scholar 

  2. Goel, R., & Gupta, P. (2020). Robotics and industry 4.0. A Roadmap to Industry 4.0: Smart Production, Sharp Business and Sustainable Development, pp. 157–169, Cham: Springer.

  3. Sheridan, T. B. (2016). Human-robot interaction: Status and challenges. Human Factors, 58(4), 525–532.

    Article  PubMed  Google Scholar 

  4. Valtazanos, A., & Ramamoorthy, S. (2013). Bayesian interaction shaping: Learning to influence strategic interactions in mixed robotic domains. AAMAS, 13, 63–70.

    Google Scholar 

  5. Kolbeinsson, A., Lagerstedt, E., & Lindblom, J. (2019). Foundation for a classification of collaboration levels for human-robot cooperation in manufacturing. Production and Manufacturing Research, 7(1), 448–471.

    Article  Google Scholar 

  6. Di, X., & Shi, R. (2021). A survey on autonomous vehicle control in the era of mixed-autonomy: From physics-based to AI-guided driving policy learning. Transportation Research Part C: Emerging Technologies, 125, 103008.

    Article  Google Scholar 

  7. Leung, K., Schmerling, E., Chen, M., Talbot, J., Gerdes, J. C., & Pavone, M. (2020). On infusing reachability-based safety assurance within probabilistic planning frameworks for human-robot vehicle interactions. In Proceedings of the 2018 International Symposium on Experimental Robotics (pp. 561–574). Springer

  8. Jin, Z., & Pagilla, P. R. (2020). Collaborative operation of robotic manipulators with human intent prediction and shared control. In IEEE International Conference on Human-Machine Systems (ICHMS) (pp. 1–6). IEEE.

  9. Sadigh, D., Landolfi, N., Sastry, S. S., Seshia, S. A., & Dragan, A. D. (2018). Planning for cars that coordinate with people: Leveraging effects on human actions for planning and active information gathering over human internal state. Autonomous Robots, 42, 1405–1426.

    Article  Google Scholar 

  10. Kavuncu, T., Yaraneri, A., & Mehr, N. (2021). Potential iLQR: A potential-minimizing controller for planning multi-agent interactive trajectories. arXiv:2107.04926

  11. Liniger, A., & Lygeros, J. (2019). A noncooperative game approach to autonomous racing. IEEE Transactions on Control Systems Technology, 28(3), 884–897.

    Article  Google Scholar 

  12. Facchinei, F., & Kanzow, C. (2010). Generalized Nash equilibrium problems. Annals of Operations Research, 175(1), 177–211.

    Article  MathSciNet  Google Scholar 

  13. Zazo, S., Macua, S. V., Sánchez-Fernández, M., & Zazo, J. (2016). Dynamic potential games with constraints: Fundamentals and applications in communications. IEEE Transactions on Signal Processing, 64(14), 3806–3821.

    Article  MathSciNet  ADS  Google Scholar 

  14. Swenson, B., Murray, R., & Kar, S. (2018). On best-response dynamics in potential games. SIAM Journal on Control and Optimization, 56(4), 2734–2767.

    Article  MathSciNet  Google Scholar 

  15. Marden, J. R., Arslan, G., & Shamma, J. S. (2009). Cooperative control and potential games. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(6), 1393–1407.

  16. Williams, Z., Chen, J., & Mehr, N. (2023). Distributed potential iLQR: Scalable game-theoretic trajectory planning for multi-agent interactions. arXiv:2303.04842

  17. Bhatt, M., Yaraneri, A., & Mehr, N. (2022). Efficient constrained multi-agent interactive planning using constrained dynamic potential games. arXiv:2206.08963.

  18. Monderer, D., & Shapley, L. S. (1996). Potential games. Games and Economic Behavior, 14(1), 124–143.

    Article  MathSciNet  Google Scholar 

  19. Fonseca-Morales, A., & Hernández-Lerma, O. (2018). Potential differential games. Dynamic Games and Applications, 8, 254–279.

    Article  MathSciNet  Google Scholar 

  20. Candogan, O., Menache, I., Ozdaglar, A., & Parrilo, P. A. (2011). Flows and decompositions of games: Harmonic and potential games. Mathematics of Operations Research, 36(3), 474–503.

    Article  MathSciNet  Google Scholar 

  21. Hatz, K., Schloder, J. P., & Bock, H. G. (2012). Estimating parameters in optimal control problems. SIAM Journal on Scientific Computing, 34(3), 1707–1728.

    Article  MathSciNet  Google Scholar 

  22. Peters, L., Fridovich-Keil, D., Rubies-Royo, V., Tomlin, C. J., & Stachniss, C. (2021). Inferring objectives in continuous dynamic games from noise-corrupted partial state observations. arXiv:2106.03611

  23. Menner, M., Worsnop, P., & Zeilinger, M. N. (2019). Constrained inverse optimal control with application to a human manipulation task. IEEE Transactions on Control Systems Technology, 29(2), 826–834.

    Article  Google Scholar 

  24. Menner, M., & Zeilinger, M. N. (2020). Maximum likelihood methods for inverse learning of optimal controllers. IFAC-PapersOnLine, 53(2), 5266–5272.

    Article  Google Scholar 

  25. Park, T., & Levine, S. (2013). Inverse optimal control for humanoid locomotion. In Robotics Science and Systems Workshop on Inverse Optimal Control and Robotic Learning from Demonstration (pp. 4887–4892).

  26. Evens, B., Schuurmans, M., & Patrinos, P. (2022). Learning MPC for interaction-aware autonomous driving: A game-theoretic approach. In 2022 European Control Conference (ECC) (pp. 34–39). IEEE.

  27. Lin, J., Wang, M., & Wu, H.-N. (2023). Composite adaptive online inverse optimal control approach to human behavior learning. Information Sciences, 638, 118977.

    Article  Google Scholar 

  28. Takei, R., Huang, H., Ding, J., & Tomlin, C. J. (2012). Time-optimal multi-stage motion planning with guaranteed collision avoidance via an open-loop game formulation. In 2012 IEEE International Conference on Robotics and Automation (pp. 323–329). IEEE.

  29. Başar, T., & Olsder, G. J. (1998). Dynamic noncooperative game theory. SIAM.

  30. Sadigh, D., Sastry, S., Seshia, S. A., & Dragan, A. D. (2016). Planning for autonomous cars that leverage effects on human actions. Robotics: Science and Systems, 2, 1–9.

  31. Fridovich-Keil, D., Ratner, E., Peters, L., Dragan, A. D., & Tomlin, C. J. (2020). Efficient iterative linear-quadratic approximations for nonlinear multi-player general-sum differential games. In 2020 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1475–1481). IEEE.

  32. Dreves, A., & Gerdts, M. (2018). A generalized Nash equilibrium approach for optimal control problems of autonomous cars. Optimal Control Applications and Methods, 39(1), 326–342.

    Article  MathSciNet  Google Scholar 

  33. Curtis, F. E., & Overton, M. L. (2012). A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization. SIAM Journal on Optimization, 22(2), 474–500.

    Article  MathSciNet  Google Scholar 

  34. Lukšan, L., Matonoha, C., & Vlček, J. (2004). Interior-point method for non-linear non-convex optimization. Numerical Linear Algebra with Applications, 11(5–6), 431–453.

    Article  MathSciNet  Google Scholar 

  35. Lin, Q., Ma, R., & Xu, Y. (2022). Complexity of an inexact proximal-point penalty method for constrained smooth non-convex optimization. Computational Optimization and Applications, 82(1), 175–224.

    Article  MathSciNet  Google Scholar 

  36. Andersson, J., Åkesson, J., & Diehl, M. (2012). CasADi: A symbolic package for automatic differentiation and optimal control. In Recent Advances in Algorithmic Differentiation (pp. 297–307). Springer.

  37. Wächter, A., & Biegler, L. T. (2006). On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, 106, 25–57.

    Article  MathSciNet  Google Scholar 

  38. Sabatino, F. (2015). Quadrotor control: Modeling, nonlinear control design, and simulation, Ph.D. Thesis. KTH, School of Electrical Engineering (EES).

  39. Maroger, I., Stasse, O., & Watier, B. (2020). Walking human trajectory models and their application to humanoid robot locomotion. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3465–3472). IEEE.

  40. Mombaur, K., Truong, A., & Laumond, J.-P. (2010). From human to humanoid locomotion-an inverse optimal control approach. Autonomous Robots, 28, 369–383.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore City, 639798, Singapore

    Xiaoxue Zhang & Lihua Xie

Authors
  1. Xiaoxue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lihua Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lihua Xie.

Additional information

This work was supported by the A\(^\star \)STAR under its “RIE2025 IAF-PP Advanced ROS2-native Platform Technologies for Cross sectorial Robotics Adoption (M21K1a0104)” programme.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Xie, L. Game-theoretic multi-agent motion planning in a mixed environment. Control Theory Technol. (2024). https://doi.org/10.1007/s11768-024-00207-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11768-024-00207-9

Keywords

Search

Navigation