Homo Egualis Reinforcement Learning Agents for Load Balancing
Periodical policies were recently introduced as a solution for the coordination problem in games which assume competition between the players, and where the overall performance can only be as good as the performance of the poorest player. Instead of converging to just one Nash equilibrium, which may favor just one of the players, a periodical policy switches between periods in which all interesting Nash equilibria are played. As a result the players are able to equalize their pay-offs and a fair solution is build. Moreover players can learn this policy with a minimum on communication; now and then they send each other their performance. In this paper, periodical policies are investigated for use in real-life asynchronous games. More precisely we look at the problem of load balancing in a simple job scheduling game. The asynchronism of the problem is reflected in delayed pay-offs or reinforcements, probabilistic job creation and processor rates which follow an exponential distribution. We show that a group of homo egualis reinforcement learning agents can still find a periodical policy. When the jobs are small, homo egualis reinforcement learning agents find a good probability distribution over their action space to play the game without any communication.
KeywordsNash Equilibrium Load Balance Action Space Common Pool Resource Learn Automaton
Unable to display preview. Download preview PDF.
- 2.Gintis, H.: Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Behavior. Princeton University Press, Princeton (2000)Google Scholar
- 4.Narendra, K., Thathachar, M.: Learning Automata: An Introduction. Prentice-Hall, Englewood Cliffs (1989)Google Scholar
- 5.Nowé, A., Parent, J., Verbeeck, K.: Social Agents Playing a Periodical Policy. In: Proceedings oft he 12th European Conference on Machine Learning, Freiburg Germany (2001) (to appear)Google Scholar
- 6.Nowé, A., Verbeeck, K.: Distributed Reinforcement learning, Loadbased Routing a case study. In: Proceedings of the Neural, Symbolic and Reinforcement Methods for sequence Learning Workshop at ijcai 1999 (1999)Google Scholar
- 8.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An introduction. MIT Press, Cambridge (1998)Google Scholar
- 9.QNAP2 reference manual, SIMULOG (1996)Google Scholar