Empowerment as a Generic Utility Function for Agents in a Simple Team Sport Simulation

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10459)


Players in team sports cooperate in a coordinated manner to achieve common goals. Automated players in academic and commercial team sports simulations have traditionally been driven by complex externally motivated value functions with heuristics based on knowledge of game tactics and strategy. Empowerment is an information-theoretic measure of an agent’s potential to influence its environment, which has been shown to provide a useful intrinsic value function, without the need for external goals and motivation, for agents in single agent models. In this paper we expand on the concept of empowerment to propose the concept of team empowerment as an intrinsic, generic utility function for cooperating agents. We show that agents motivated by team empowerment exhibit recognizable team behaviors in a simple team sports simulation based on Ultimate Frisbee.


Empowerment Intrinsic motivation Artificial intelligence Information theory 



DP would like to acknowledge support by the EC H2020-641321 socSMCs FET Proactive 713 project, the H2020-645141 WiMUST ICT-23-2014 Robotics project.


  1. 1.
    Anthony, T., Polani, D., Nehaniv, C.L.: General self-motivation and strategy identification: case studies based on Sokoban and Pac-Man. IEEE Trans. Comput. Intell. AI Games 6(1), 1–17 (2014)CrossRefGoogle Scholar
  2. 2.
    Ay, N., Bertschinger, N., Der, R., Güttler, F., Olbrich, E.: Predictive information and explorative behavior of autonomous robots. Eur. Phys. J. B-Condens. Matter Complex Syst. 63(3), 329–339 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Barto, A.G., Mahadevan, S.: Recent advances in hierarchical reinforcement learning. Discret. Event Dyn. Syst. 13(4), 341–379 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Capdepuy, P.: Informational principles of perception-action loops and collective behaviors. Ph.D. thesis, University of Hertfordshire (2010)Google Scholar
  5. 5.
    Capdepuy, P., Polani, D., Nehaniv, C.: Maximization of potential information flow as a universal utility for collective behavior. In: IEEE Symposium on Artificial Life, pp. 207–213. IEEE (2007)Google Scholar
  6. 6.
    Capdepuy, P., Polani, D., Nehaniv, C.L.: Perception-action loops of multiple agents: informational aspects and the impact of coordination. Theory Biosci. 131(3), 149–159 (2012)CrossRefGoogle Scholar
  7. 7.
    Csikszentmihalyi, M.: Beyond Boredom and Anxiety. Jossey-Bass, San Francisco (2000)Google Scholar
  8. 8.
    Der, R., Steinmetz, U., Pasemann, F.: Homeokinesis: a new principle to back up evolution with learning. Max-Planck-Institut für Mathematik in den Naturwiss (1999)Google Scholar
  9. 9.
    Duarte, R., Duarte, A., Correia, V., Davids, K.: Sports teams as superorganisms. Sports Med. 42(8), 633–642 (2012)CrossRefGoogle Scholar
  10. 10.
    Dylla, F., Ferrein, A., Lakemeyer, G., Murray, J., Obst, O., Röfer, T., Schiffer, S., Stolzenburg, F., Visser, U., Wagner, T.: Approaching a formal soccer theory from the behavior specification in robotic soccer. In: Dabnichki, P., Baca, A. (eds.) Computers in Sport, pp. 161–186. WIT Press, London (2008)CrossRefGoogle Scholar
  11. 11.
    Jung, T., Polani, D., Stone, P.: Empowerment for continuous agent-environment systems. Adapt. Behav. 19(1), 16–39 (2011)CrossRefGoogle Scholar
  12. 12.
    Karl, M., Bayer, J., van der Smagt, P.: Efficient empowerment (2015, preprint)Google Scholar
  13. 13.
    Kelly, K.: Out of Control: The New Biology of Machines, Social Systems and the Economic World. Perseus, Cambridge (1994)Google Scholar
  14. 14.
    Klyubin, A., Polani, D., Nehaniv, C.: Empowerment: a universal agent-centric measure of control. In: The 2005 IEEE Congress on Evolutionary Computation, vol. 1, pp. 128–135 (2005)Google Scholar
  15. 15.
    Klyubin, A., Polani, D., Nehaniv, C.: Keep your options open: an information-based driving principle for sensorimotor systems. PloS ONE 3, 12 (2008)CrossRefGoogle Scholar
  16. 16.
    Klyubin, A.S., Polani, D., Nehaniv, C.L.: All else being equal be empowered. In: Capcarrère, M.S., Freitas, A.A., Bentley, P.J., Johnson, C.G., Timmis, J. (eds.) ECAL 2005. LNCS, vol. 3630, pp. 744–753. Springer, Heidelberg (2005). doi: 10.1007/11553090_75 CrossRefGoogle Scholar
  17. 17.
    Mohamed, S., Rezende, D.J.: Variational information maximisation for intrinsically motivated reinforcement learning. In: Neural Information Processing Systems (NIPS) 2015 (2015)Google Scholar
  18. 18.
    Salge, C., Glackin, C., Polani, D.: Empowerment–an introduction. In: Prokopenko, M. (ed.) Guided Self-Organization: Inception. ECC, vol. 9, pp. 67–114. Springer, Heidelberg (2014). doi: 10.1007/978-3-642-53734-9_4 CrossRefGoogle Scholar
  19. 19.
    Samuelson, L.: Analogies, adaptation, and anomalies. J. Econ. Theory 97(2), 320–366 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Schmidhuber, J.: Curious model-building control systems. In: IEEE International Joint Conference on Neural Networks, pp. 1458–1463. IEEE (1991)Google Scholar
  21. 21.
    Schmidhuber, J.: Exploring the predictable. Adv. Evol. Comput. 6, 579–612 (2002)Google Scholar
  22. 22.
    Schmidhuber, J.: Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE Trans. Auton. Mental Dev. 2(3), 230–247 (2010)CrossRefGoogle Scholar
  23. 23.
    Shannon, C.E.: A mathematical theory of communication. Bell Sys. Tech. J. 27, 623–656 (1948)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Steels, L.: The autotelic principle. In: Embodied Artificial Intelligence: International Seminar, Dagstuhl Castle, Germany, 7–11 July 2003, pp. 231–242 (2004). (Revised Papers)Google Scholar
  25. 25.
    Stone, P.: Layered Learning in Multiagent Systems: A Winning Approach to Robotic Soccer. MIT Press, Cambridge (1998)Google Scholar
  26. 26.
    World Flying Disc Federation Ultimate Rules Committee: WFDF Rules of Ultimate 2017 (2017).

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Adaptive Systems Research Group, School of Computer ScienceUniversity of HertfordshireHatfieldUK

Personalised recommendations