Skip to main content
SpringerLink
Log in

Graphical Games: Distributed Multiplayer Games on Graphs

  • Chapter
  • First Online:
Cooperative Control of Multi-Agent Systems

Part of the book series: Communications and Control Engineering ((CCE))

  • 5628 Accesses

Abstract

In this chapter, it is seen that distributed control protocols that both guarantee synchronization and are globally optimal for the multi-agent team always exist on any sufficiently connected communication graph if a different definition of optimality is used. To this end, we study the notion of Nash equilibrium for multiplayer games on graphs. This leads us to the idea of a new sort of differential game—graphical games. In graphical games, each agent has its own dynamics as well as its own local performance index. The dynamics and local performance indices of each agent are distributed; they depend on the state of the agent, the control of the agent, and the controls of the agent’s neighbors. We show how to compute distributed control protocols that guarantee global Nash equilibrium for multi-agent teams on any graph that has a spanning tree.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abou-Kandil H, Freiling G, Ionescu V, Jank G (2003) Matrix Riccati Equations in Control and Systems Theory. Birkhäuser

    Google Scholar 

  2. Abu-Khalaf M, Lewis FL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5):779–791

    Google Scholar 

  3. Al-Tamimi A, Abu-Khalaf M, Lewis FL (2007) Adaptive critic designs for discrete-time zero-sum games with application to H-Infinity control. IEEE Trans Syst, Man, Cybern B 37(1):240–247

    Google Scholar 

  4. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst, Man, Cybern B 38(4):943–949

    Google Scholar 

  5. Başar T, Olsder GJ (1999) Dynamic Noncooperative Game Theory, 2nd edn. SIAM, Philadelphia

    Google Scholar 

  6. Bertsekas DP, Tsitsiklis JN (1996) Neuro-Dynamic Programming. Athena Scientific, Belmont

    Google Scholar 

  7. Brewer JW (1978) Kronecker products and matrix calculus in system theory. IEEE Trans Circuits Syst 25:772–781

    Google Scholar 

  8. Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multi-agent reinforcement learning. IEEE Trans Syst, Man, Cybern C 38(2):156–172

    Google Scholar 

  9. Dierks T, Jagannathan S (2010) Optimal control of affine nonlinear continuous-time systems using an online Hamilton–Jacobi–Isaacs formulation. In: Proc. IEEE Conf. Decision Control, Atlanta, GA, pp. 3048–3053

    Google Scholar 

  10. Freiling G, Jank G, Abou-Kandil H (2002) On global existence of solutions to coupled matrix Riccati equations in closed loop Nash games. IEEE Trans Automat Contr 41(2):264–269

    Google Scholar 

  11. Gajic Z, Li T-Y (1988) Simulation results for two new algorithms for solving coupled algebraic Riccati equations. Paper presented at 3rd international symposium on differential games, Sophia Antipolis, Nice, France

    Google Scholar 

  12. Goldberg AV (1995) Scaling algorithms for the shortest paths problem. SIAM J Comput 24:494–504

    Google Scholar 

  13. Ioannou P, Fidan B (2006) Adaptive Control Tutorial. SIAM, Philadelphia

    Google Scholar 

  14. Johnson M, Hiramatsu T, Fitz-Coy N, Dixon WE (2010) Asymptotic stackelberg optimal control design for an uncertain euler lagrange system. In: Proc. IEEE Conf. Decision Control, Atlanta, GA, pp. 6686–6691

    Google Scholar 

  15. Kakade S, Kearns M, Langford J, Ortiz L (2003) Correlated equilibria in graphical games. In: the 4th ACM conf. Electron. Commerce, San Diego, CA, pp. 42–47

    Google Scholar 

  16. Kearns M, Littman M, Singh S (2001) Graphical models for game theory. In: Proc. Annual conf. Uncertainty in Artificial Intelligence, Seattle, WA, pp. 253–260

    Google Scholar 

  17. Khoo S, Xie L, Man Z (2009) Robust finite-time consensus tracking algorithm for multirobot systems. IEEE Trans Mechatron 14:219–228

    Google Scholar 

  18. Leake RJ, Liu R-W (1967) Construction of suboptimal control sequences. SIAM J Contr 5(1):54–63

    Google Scholar 

  19. Lewis FL (1992) Applied Optimal Control and Estimation: Digital Design and Implementation. Prentice-Hall, Upper Saddle River

    Google Scholar 

  20. Lewis FL, Vrabie D (2009) Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits & Systems Magazine (invited feature article), pp. 32–50, Third Quarter 2009

    Google Scholar 

  21. Lewis FL, Jagannathan S, Yesildirek A (1999) Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor and Francis, London

    Google Scholar 

  22. Lewis FL, Vrabie D, Syrmos VL (2012) Optimal control, 3rd edn. Wiley, Hoboken

    Google Scholar 

  23. Lewis FL, Vrabie D, Vamvoudakis KG (2012) Reinforcement learning and feedback control. IEEE Control Systems Magazine, pp. 76–105

    Google Scholar 

  24. Li X, Wang X, Chen G (2004) Pinning a complex dynamical network to its equilibrium. IEEE Trans Circuits Syst I, Reg Papers 51(10):2074–2087

    Google Scholar 

  25. Littman ML (2001) Value-function reinforcement learning in Markov games. J Cogn Syst Res 2(1):55–66

    Google Scholar 

  26. Marden JR, Young HP, Pao LY (2012) Achieving pareto optimality through distributed learning. In: Proc. IEEE Conf. Decision Control, Maui, HI, pp. 7419–7424

    Google Scholar 

  27. Shinohara R (2010) Coalition proof equilibria in a voluntary participation game. Int J Game Theory 39(4):603–615

    Google Scholar 

  28. Shoham Y, Leyton-Brown K (2009). Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge

    Google Scholar 

  29. Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, Cambridge

    Google Scholar 

  30. Tijs S (2003) Introduction to game theory. Hindustan Book Agency, New Delhi.

    Google Scholar 

  31. Vamvoudakis KG, Lewis FL (2010). Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    Google Scholar 

  32. Vamvoudakis KG, Lewis FL (2011). Multi-player non-zero sum games: online adaptive learning solution of coupled Hamilton–Jacobi equations. Automatica 47(8):1556–1569

    Google Scholar 

  33. Vamvoudakis KG, Lewis FL, Hudas GR (2012) Multi-agent differential graphical games: online adaptive learning solution for synchronization with optimality. Automatica 48(8):1598–1611

    Google Scholar 

  34. Vrabie D, Lewis FL (2009) Neural network approach to continuous-time direct adaptive optimal control for partially-unknown nonlinear systems. Neural Networks 2(3):237–246

    Google Scholar 

  35. Vrabie D, Pastravanu O, Lewis FL, Abu-Khalaf M (2009). Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45(2):477–484

    Google Scholar 

  36. Vrancx P, Verbeeck K, Nowe A (2008). Decentralized learning in Markov games. IEEE Tran Syst Man Cyber 38(4):976–981

    Google Scholar 

  37. Wang F, Zhang H, Liu D (May 2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39–47

    Google Scholar 

  38. Wang X, Chen G (2002). Pinning control of scale-free dynamical networks. Physica A 310(3–4):521–531

    Google Scholar 

  39. Werbos PJ (1974) Beyond Regression: New Tools for Prediction and Analysis in the Behavior Sciences. Ph.D. Thesis, Harvard University

    Google Scholar 

  40. Werbos PJ (1992) Approximate dynamic programming for real-time control and neural modeling. In: White DA, Sofge DA (eds) Handbook of Intelligent Control. Van Nostrand Reinhold, New York

    Google Scholar 

  41. Zwick U (2002) All pairs shortest paths using bridging sets and rectangular matrix multiplication. J ACM 49(3):289-317.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. UTA Research Institute, University of Texas at Arlington, Fort Worth, Texas, USA

    Frank L. Lewis

  2. School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China, People’s Republic

    Hongwei Zhang

  3. UTA Research Institute, University of Texas at Arlington, Fort Worth, Texas, USA

    Kristian Hengster-Movric

  4. Advanced Systems Engineering​, Danfoss Power Solutions (US) Company​, Ames, IA, USA

    Abhijit Das

Authors
  1. Frank L. Lewis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kristian Hengster-Movric
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abhijit Das
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank L. Lewis .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Lewis, F., Zhang, H., Hengster-Movric, K., Das, A. (2014). Graphical Games: Distributed Multiplayer Games on Graphs. In: Cooperative Control of Multi-Agent Systems. Communications and Control Engineering. Springer, London. https://doi.org/10.1007/978-1-4471-5574-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5574-4_6

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5573-7

  • Online ISBN: 978-1-4471-5574-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation