Applying Reinforcement Learning in Formation Control of Agents
This paper proposes a new Reinforcement Learning (RL) algorithm for formation of agents in regular geometric forms. Due to curse of dimensionality problem, applying RL algorithms in formation problems cannot present suitable performance. Moreover, since the state space in formation problem is large, this leads to long learning time. Here, a multi-agent fuzzy reinforcement learning algorithm is presented that is an extension of fuzzy actor-critic reinforcement learning in a multi-agent environment. The final action for each agent is generated by a zero order T-S fuzzy system. In conventional fuzzy actor-critic RL, there are several candidate actions for consequence of each fuzzy rule and aim of learning is finding the best action among these discrete candidate actions. Here, using the proposed linear interpolation, a continuous action selection for determining the best action for each fuzzy rule is presented. The simulation results show the proposed method can improve the learning speed and action quality.
This research was financially supported by the Center of Excellence for Robust and Intelligence Systems (CERIS) of Yazd University.
- 2.Breivik, M.: Topics in guided motion control of marine vehicles. Ph.D. thesis, Norwegian University of Science and Technology (2010)Google Scholar
- 3.Chen, G., Cao, W., Chen, X., Wu, M.: Multi-agent q-learning with joint state value approximation. In: Proceedings of the 30th Chinese Control Conference, p. 48784882 (2011)Google Scholar
- 6.Dierks, T., Jagannathan, S.: Neural network output feedback control of robot formations. IEEE Trans. Syst., Man, Cybern. Part B 40, pp. 383-399 (2010)Google Scholar
- 8.Franco, F.E., Waissman, V.J., Garca, L.J.: Learning the filling policy of a biodegradation process by fuzzy actorcritic learning methodology. Adv. Artif. Intell. 5317, 243–253 (2008)Google Scholar
- 9.Izzo, D., Pettazzi, L.: Autonomous and distributed motion planning for satellite swarm. J. Guid. Control Dyn. 30, 449459 (2005)Google Scholar
- 10.Macdonald, E.A.: Multi-robot assignment and formation control. M.Sc. thesis, Georgia Institute of Technology (2011)Google Scholar
- 12.Sanz, Y., de Lope, J., Martn, J.A.H.: Applying reinforcement learing to multi-robot team coordination. In: Corchado, E., Abraham, A., Pedrycz, W. (eds.) HAIS 2008. LNCS, vol. 5271, pp. 625632. Springer, Heidelberg (2008)Google Scholar
- 13.Zuo, G., Han, J., Han, G.: Multi-robot formation control using reinforcement learning, advances in swarm Intelligence. vol. 6145, pp. 667–674. Springer, Berlin (2010)Google Scholar