Abstract
Reactive multi-agent systems present global behaviours uneasily linked to their local dynamics. When it comes to controlling such a system, usual analytical tools are difficult to use so specific techniques have to be engineered. We propose an experimental dynamical approach to enhance the control of the global behaviour of a reactive multi-agent system. We use reinforcement learning tools to link global information of the system to control actions. We propose to use the behaviour of the system as this global information. The behaviour of the whole system is controlled thanks to actions at different levels instead of building the behaviours of the agents, so that the complexity of the approach does not directly depend on the number of agents. The controllability is evaluated in terms of rate of convergence towards a target behaviour. We compare the results obtained on a toy example with the usual approach of parameter setting.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ferber, J.: Multi-Agent System: An Introduction to Distributed Artificial Intelligence. Addison Wesley Longman, Harlow (1999)
Wegner, P.: Why interaction is more powerful than algorithms. Communications of the ACM 40, 80–91 (1997)
Edmonds, B.: Using the Experimental Method to Produce Reliable Self-Organised Systems. In: Engineering Self Organising Sytems: Methodologies and Applications, Springer, Heidelberg (2004)
Edmonds, B., Bryson, J.: The Insufficiency of Formal Design Methods - the necessity of an experimental approach for the understanding and control of complex MAS. In: Proceedings of the 3rd International Joint AAMAS 2004, pp. 938–945. ACM Press, New York (2004)
De Wolf, T., Holvoet, T.: Towards a Methodology for Engineering Self-Organising Emergent Systems. In: Proceedings of SOAS 2005, Glasgow, Scotland (2005)
Amblard, F.: Comprendre le fonctionnement de simulations sociales individus-centrées. Thèse de doctorat en Informatique, Université Clermont II (2003)
Sauter, J.A., Parunak, H.V.D., Brueckner, S., Matthews, R.: Tuning Synthetic Pheromones Withe Evolutionary Computing. In: Genetic and Evolutionary Computation Conference Workshop Program (GECCO 2001), San Fransisco, CA (2001)
Sierra, C., Sabater, J., Agusti, J., Garcia, P.: Evolutionary Computation in MAS Design. In: Proceedings ECAI, pp. 188–192 (2002)
Dréo, J., Petrowski, A., Taillard, E., Siarry, P.: Metaheuristics for Hard Optimization Methods and Case Studies. Springer, Heidelberg (2006)
De Wolf, T., Samaey, G., Holvoet, T.: Engineering Self-Organising Emergent Systems with Simulation-based Scientific Analysis. In: Brueckner, S., Di Marzo Serugendo, G., Hales, D., Zambonelli, F. (eds.) Proceedings of the Third International Workshop on Engineering Self-Organising Applications, Utrecht, The Netherlands, pp. 146–160 (2005)
Fehler, M., Klügl, F., Puppe, F.: Approaches for resolving the dilemma between model structure refinement and parameter calibration in agent-based simulations. In: AAMAS 2006, pp. 120–122 (2006)
Calvez, B., Hutzler, G.: Automatic tuning of agent-based models using genetic algorithms. In: Sichman, J.S., Antunes, L. (eds.) MABS 2005. LNCS(LNAI), vol. 3891, pp. 41–57. Springer, Heidelberg (2006)
Narzisi, G., Mysore, V., Bud Mishra, B.: Multi-objective evolutionary optimization of agent-based models: An application to emergency response planning. In: Kovalerchuk, B. (ed.) The IASTED International Conference on Computational Intelligence, CI 2006 (2006)
Klein, F., Bourjot, C., Chevrier, V.: Approche expérimentale pour la compréhension des systèmes multi-agents réactifs. In: JFSMA 2006, Annecy (2006)
Calvez, B., Hutzler, G.: Ant Colony Systems and the Calibration of Multi-Agent Simulations: a New Approach. In: MA4CS 2007 Satellite Workshop of ECCS 2007 (2007)
Brueckner, S., Van Dyke Parunak, H.: Resource-aware exploration of the emergent dynamics of simulated systems. In: AAMAS 2003, pp. 781–788 (2003)
Campagne, J.C., Cardon, A., Collomb, E., Nishida, T.: Using morphology to analyse and control a Multi-Agent system, an example. In: STAIRS ECAI 2004 (August 2004)
Campagne, J.-C., Cardon, A., Collomb, E., Nishida, T.: Massive multi-agent systems control. In: Hinchey, M.G., Rash, J.L., Truszkowski, W.F., Rouff, C.A. (eds.) FAABS 2004. LNCS, vol. 3228, pp. 275–280. Springer, Heidelberg (2004)
Bernon, C., Camps, V., Gleizes, M.-P., Picard, G.: Engineering Adaptive Multi-Agent Systems: the ADELFE Methodology. In: Henderson-Sellers, B., Giorgini, P. (eds.) Agent-Oriented Methodologies, June 2005, pp. 172–202. Idea Group Pub. (2005)
Bernon, C., Gleizes, M.-P., Picard, G.: Enhancing Self-Organising Emergent Systems Design with Simulation. In: O’Hare, G.M.P., Ricci, A., O’Grady, M.J., Dikenelli, O. (eds.) ESAW 2006. LNCS, vol. 4457, pp. 284–299. Springer, Heidelberg (2007)
Thomas, V., Bourjot, C., Chevrier, V.: Interac-DEC-MDP: Towards the use of interactions in DEC-MDP. In: Third International Joint Conference on Autonomous Agents and Multi-Agent Systems - AAMAS 2004, New York, USA, pp. 1450–1451 (2004)
Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S.: The complexity of decentralized control of markov decision processes. Mathematics of Operations Research 27(4), 819–840 (2002)
Sutton, R., Barto, A.: Reinforcement Learning: an introduction. MIT Press, Cambridge (1998)
Craig Reynolds’ boids; http://www.red3d.com/cwr/index.html
Lacroix, B., Mathieu, P., Picault, S.: Time and Space Management in Crowd Simulations. In: Proceedings of the European Simulation and Modelling Conference (ESM 2006), Toulouse, France, pp. 315–320 (2006)
Jain, A.K., Murty, M.N., Flynn, P.J.: Data Clustering: A Review. ACM Computer Survey 31(3), 264–323 (1999)
Handl, J., Knowles, J.: Multiobjective clustering with automatic determination of the number of clusters. In: Technical Report TR-COMPSYSBIO-2004-02. UMIST, Manchester (2004)
Scerri, P., Pynadath, D.V., Tambe, M.: Towards Adjustable Autonomy for the Real World. J. Artif. Intell. Res. (JAIR) 17, 171–228 (2002)
Van Hasselt, H., Wiering, M.: Reinforcement Learning in Continuous Action Spaces. In: Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, USA, pp. 272–279 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Klein, F., Bourjot, C., Chevrier, V. (2009). Contribution to the Control of a MAS’s Global Behaviour: Reinforcement Learning Tools. In: Artikis, A., Picard, G., Vercouter, L. (eds) Engineering Societies in the Agents World IX. ESAW 2008. Lecture Notes in Computer Science(), vol 5485. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02562-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-02562-4_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02561-7
Online ISBN: 978-3-642-02562-4
eBook Packages: Computer ScienceComputer Science (R0)