Chapter

Adaptive Agents and Multi-Agent Systems III. Adaptation and Multi-Agent Learning

Volume 4865 of the series Lecture Notes in Computer Science pp 129-144

Bifurcation Analysis of Reinforcement Learning Agents in the Selten’s Horse Game

  • Alessandro LazaricAffiliated withDepartment of Electronics and Information, Politecnico di Milano
  • , Enrique Munoz de CoteAffiliated withDepartment of Electronics and Information, Politecnico di Milano
  • , Fabio DercoleAffiliated withDepartment of Electronics and Information, Politecnico di Milano
  • , Marcello RestelliAffiliated withDepartment of Electronics and Information, Politecnico di Milano

* Final gross prices may vary according to local VAT.

Get Access

Abstract

The application of reinforcement learning algorithms to multiagent domains may cause complex non-convergent dynamics. The replicator dynamics, commonly used in evolutionary game theory, proved to be effective for modeling the learning dynamics in normal form games. Nonetheless, it is often interesting to study the robustness of the learning dynamics when either learning or structural parameters are perturbed. This is equivalent to unfolding the catalog of learning dynamical scenarios that arise for all possible parameter settings which, unfortunately, cannot be obtained through “brute force” simulation of the replicator dynamics. The analysis of bifurcations, i.e., critical parameter combinations at which the learning behavior undergoes radical changes, is mandatory. In this work, we introduce a one-parameter bifurcation analysis of the Selten’s Horse game in which the learning process exhibits a set of complex dynamical scenarios even for relatively small perturbations on payoffs.