Game-Theoretic Learning in Distributed Control

  • Jason R. MardenEmail author
  • Jeff S. Shamma
Living reference work entry


In distributed architecture control problems, there is a collection of interconnected decision-making components that seek to realize desirable collective behaviors through local interactions and by processing local information. Applications range from autonomous vehicles to energy to transportation. One approach to control of such distributed architectures is to view the components as players in a game. In this approach, two design considerations are the components’ incentives and the rules that dictate how components react to the decisions of other components. In game-theoretic language, the incentives are defined through utility functions, and the reaction rules are online learning dynamics. This chapter presents an overview of this approach, covering basic concepts in game theory, special game classes, measures of distributed efficiency, utility design, and online learning rules, all with the interpretation of using game theory as a prescriptive paradigm for distributed control design.


Learning in games Evolutionary games Multiagent systems Distributed decision systems 


  1. Alos-Ferrer C, Netzer N (2010) The logit-response dynamics. Games Econ Behav 68:413–427MathSciNetCrossRefzbMATHGoogle Scholar
  2. Altman E, Bonneau N, Debbah M (2006) Correlated equilibrium in access control for wireless communications. In: 5th International Conference on NetworkingGoogle Scholar
  3. Babichenko Y (2012) Completely uncoupled dynamics and Nash equilibria. Games Econ Behav 76:1–14MathSciNetCrossRefzbMATHGoogle Scholar
  4. Blondel VD, Hendrickx JM, Olshevsky A, Tsitsiklis JN (2005a) Convergence in multiagent coordination, consensus, and flocking. In: IEEE Conference on Decision and Control, pp 2996–3000Google Scholar
  5. Blondel VD, Hendrickx JM, Olshevsky A, Tsitsiklis JN (2005b) Convergence in multiagent coordination, consensus, and flocking. In: Proceedings of the Joint 44th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC’05), SevilleGoogle Scholar
  6. Blume L (1993) The statistical mechanics of strategic interaction. Games Econ Behav 5:387–424MathSciNetCrossRefzbMATHGoogle Scholar
  7. Blume L (1997) Population games. In: Arthur B, Durlauf S, and Lane D (eds) The economy as an evolving complex system II. Addison-Wesley, Reading, pp 425–460Google Scholar
  8. Borowski H, Marden JR, Frew EW (2013) Fast convergence in semi-anonymous potential games. In: Proceedings of the IEEE Conference on Decision and Control, pp 2418–2423Google Scholar
  9. Borowski HP, Marden JR, Shamma JS (2014) Learning efficient correlated equilibria. In: Proceedings of the IEEE Conference on Decision and Control, pp 6836–6841Google Scholar
  10. Cortes J, Martinez S, Karatas T, Bullo F (2002) Coverage control for mobile sensing networks. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’02), Washington, DC, pp 1327–1332Google Scholar
  11. Daskalakis C, Goldberg PW, Papadimitriou CH (2009) The complexity of computing a Nash equilibrium. SIAM J Comput 39(1):195–259MathSciNetCrossRefzbMATHGoogle Scholar
  12. Fudenberg D, Levine DK (1995) Consistency and cautious fictitious play. Games Econ Behav 19:1065–1089MathSciNetzbMATHGoogle Scholar
  13. Fudenberg D, Levine DK (1998) The theory of learning in games. MIT Press, CambridgezbMATHGoogle Scholar
  14. Fudenberg D, Tirole J (1991) Game theory. MIT Press, CambridgezbMATHGoogle Scholar
  15. Gopalakrishnan R, Marden JR, Wierman A (2014) Potential games are necessary to ensure pure Nash equilibria in cost sharing games. Math Oper Res 39(4):1252–1296MathSciNetCrossRefzbMATHGoogle Scholar
  16. Hart S (2005) Adaptive heuristics. Econometrica 73(5):1401–1430MathSciNetCrossRefzbMATHGoogle Scholar
  17. Hart S, Mansour Y (2010) How long to equilibrium? The communication complexity of uncoupled equilibrium procedures. Games Econ Behav 69(1):107–126MathSciNetCrossRefzbMATHGoogle Scholar
  18. Hart S, Mas-Colell A (2000) A simple adaptive procedure leading to correlated equilibrium. Econometrica 68(5):1127–1150MathSciNetCrossRefzbMATHGoogle Scholar
  19. Hart S, Mas-Colell A (2003) Uncoupled dynamics do not lead to Nash equilibrium. Am Econ Rev 93(5):1830–1836CrossRefGoogle Scholar
  20. Jadbabaie A, Lin J, Morse AS (2003) Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Trans Autom Control 48(6):988–1001MathSciNetCrossRefzbMATHGoogle Scholar
  21. Kearns MJ, Littman ML, Singh SP (2001) Graphical models for game theory. In: Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, pp 253–260Google Scholar
  22. Lambert III TJ, Epelman MA, Smith RL (2005) A fictitious play approach to large-scale optimization. Oper Res 53(3):477–489MathSciNetCrossRefzbMATHGoogle Scholar
  23. Marden JR (2012) State based potential games. Automatica 48:3075–3088MathSciNetCrossRefzbMATHGoogle Scholar
  24. Marden JR (2015) Selecting efficient correlated equilibria through distributed learning. In: American Control Conference, pp 4048–4053Google Scholar
  25. Marden JR, Shamma JS (2012) Revisiting log-linear learning: asynchrony, completeness and a payoff-based implementation. Games Econ Behav 75(2):788–808MathSciNetCrossRefzbMATHGoogle Scholar
  26. Marden JR, Shamma JS (2015) Game theory and distributed control. In: Young HP, Zamir S (eds) Handbook of game theory with economic applications, vol 4. Elsevier Science, pp 861–899Google Scholar
  27. Marden JR, Wierman A (2013) Distributed welfare games. Oper Res 61:155–168MathSciNetCrossRefzbMATHGoogle Scholar
  28. Marden JR, Arslan G, Shamma JS (2009) Joint strategy fictitious play with inertia for potential games. IEEE Trans Autom Control 54:208–220MathSciNetCrossRefzbMATHGoogle Scholar
  29. Martinez S, Cortes J, Bullo F (2007) Motion coordination with distributed information. Control Syst Mag 27(4):75–88CrossRefGoogle Scholar
  30. Monderer D, Shapley LS (1996) Fictitious play property for games with identical interests. J Econ Theory 68:258–265MathSciNetCrossRefzbMATHGoogle Scholar
  31. Montanari A, Saberi A (2009) Convergence to equilibrium in local interaction games. In: 50th Annual IEEE Symposium on Foundations of Computer ScienceGoogle Scholar
  32. Murphey RA (1999) Target-based weapon target assignment problems. In: Pardalos PM, Pitsoulis LS (eds) Nonlinear assignment problems: algorithms and applications. Kluwer Academic, AlexandraGoogle Scholar
  33. Nisan N, Roughgarden T, Tardos E, Vazirani VV (2007) Algorithmic game theory. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  34. Olfati-Saber R (2006) Flocking for multi-agent dynamic systems: algorithms and theory. IEEE Trans Autom Control 51:401–420MathSciNetCrossRefzbMATHGoogle Scholar
  35. Olfati-Saber R, Murray RM (2003) Consensus problems in networks of agents with switching topology and time-delays. IEEE Trans Autom Control 49(9):1520–1533MathSciNetCrossRefzbMATHGoogle Scholar
  36. Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. Proc IEEE 95(1):215–233CrossRefzbMATHGoogle Scholar
  37. Roughgarden T (2005) Selfish routing and the price of anarchy. MIT Press, CambridgezbMATHGoogle Scholar
  38. Roughgarden T (2015) Intrinsic robustness of the price of anarchy. J ACM 62(5):32:1–32:42Google Scholar
  39. Shah D, Shin J (2010) Dynamics in congestion games. In: ACM SIGMETRICS, pp 107–118Google Scholar
  40. Shamma JS (2014) Learning in games. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, LondonGoogle Scholar
  41. Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365–377MathSciNetCrossRefzbMATHGoogle Scholar
  42. Touri B, Nedic A (2011) On ergodicity, infinite flow, and consensus in random models. IEEE Trans Autom Control 56(7):1593–1605MathSciNetCrossRefzbMATHGoogle Scholar
  43. Tsitsiklis JN (1987) Decentralized detection by a large number of sensors. Technical report. MIT, LIDSzbMATHGoogle Scholar
  44. Tsitsiklis JN, Bertsekas DP, Athans M (1986) Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Trans Autom Control 35(9):803–812MathSciNetCrossRefzbMATHGoogle Scholar
  45. A. Vetta, “Nash equilibria in competitive societies, with applications to facility location, traffic routing and auctions,” Proceedings of the 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002, pp 416–425Google Scholar
  46. Wang B, Han Z, Liu KJR (2009) Peer-to-peer file sharing game using correlated equilibrium. In: 43rd Annual Conference on Information Sciences and Systems, CISS 2009, pp 729–734Google Scholar
  47. Wolpert D, Tumor K (1999) An overview of collective intelligence. In: Bradshaw JM (ed) Handbook of agent technology. AAAI Press/MIT Press, Cambridge, USAGoogle Scholar
  48. Young HP (1998) Individual strategy and social structure. Princeton University Press, PrincetonGoogle Scholar
  49. Young HP (2004) Strategic learning and its limits. Oxford University Press, New YorkCrossRefGoogle Scholar
  50. Zhu M, Martínez S (2013) Distributed coverage games for energy-aware mobile sensor networks. SIAM J Control Optim 51(1):1–27MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of CaliforniaSanta BarbaraUSA
  2. 2.Computer, Electrical and Mathematical Science and Engineering Division (CEMSE)King Abdullah University of Science and Technology (KAUST)ThuwalSaudi Arabia

Section editors and affiliations

  • Tamer Başar
    • 1
  • Georges Zaccour
    • 2
  1. 1.Coordinated Science LaboratoryUniversity of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.Département de sciences de la décisionGERAD, HEC MontréalMontrealCanada

Personalised recommendations