KI - Künstliche Intelligenz

, Volume 28, Issue 3, pp 179–189 | Cite as

Beyond Reinforcement Learning and Local View in Multiagent Systems

Technical Contribution

Abstract

Learning is an important component of an agent’s decision making process. Despite many messages in contrary, the fact is that, currently, in the multiagent community it is mostly likely that learning means reinforcement learning. Given this background, this paper has two aims: to revisit the “old days” motivations for multiagent learning, and to describe some of the work addressing the frontiers of multiagent systems and machine learning. The intention of the latter task is to try to motivate people to address the issues that are involved in the application of techniques from multiagent systems in machine learning and vice-versa.

Keywords

Multiagent systems Multiagent learning Machine learning Distributed machine learning Reinforcement learning 

References

  1. 1.
    Agogino A, Tumer K (2006) Efficient agent-based cluster ensembles. In: Stone P, Weiss G (eds) AAMAS ’06: Proceedings of the 5th International Joint Conference on Autonomous agents and Multiagent Systems. ACM, New York, pp 1079–1086CrossRefGoogle Scholar
  2. 2.
    Alam S, Dobbie G, Riddle P (2009) Exploiting swarm behaviour of simple agents for clustering web users’ session data. In: Cao L (ed) Data mining and multi-agent integration, pp 61–75. Springer, US. doi:10.1007/978-1-4419-0522-2-4
  3. 3.
    Babu T, Murty M, Subrahmanya S (2009) Multiagent systems for large data clustering. In: Cao L (ed) Data mining and multi-agent integration. pp 219–238. Springer, US. doi:10.1007/978-1-4419-0522-2-15
  4. 4.
    Bazzan ALC (2009) Agents and data mining in bioinformatics: joining data gathering and automatic annotation with classification and distributed clustering. In: Cao L (ed) Proceedings of the Workshop on Agents and Data Mining Interaction, no. 5680 in Lecture Notes in Artificial Intelligence, pp 3–20. Springer, Berlin. URL http://www.inf.ufrgs.br/bazzan/downloads/5680_3.pdf
  5. 5.
    Bazzan ALC (2013) Cooperative induction of decision trees. In: 2013 IEEE Symposium on Intelligent Agent (IA) pp 62–69. doi:10.1109/IA.2013.6595190
  6. 6.
    Bekkerman R, Zilberstein S, Allan J (2007) Web page clustering using heuristic search in the web graph. In: Veloso MM (ed) IJCAI, pp 2280–2285Google Scholar
  7. 7.
    Brahmi I, Yahia S, Aouadi H, Poncelet P (2012) Towards a multiagent-based distributed intrusion detection system using data mining approaches. In: Cao L, Bazzan AL, Symeonidis AL, Gorodetsky VI, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7103, pp 173–194. Springer, Berlin. doi:10.1007/978-3-642-27609-5-12
  8. 8.
    Brazdil P, Gams M, Sian SS, Torgo L, de Velde WV (1991) Panel: learning in distributed systems and multi-agent environments. In: Kodratoff Y (ed) European Working Session on Learning (EWSL-91), Lecture Notes in Computer Science, vol. 482, pp 412–423. Springer. URL http://dblp.uni-trier.de/db/conf/ecml/ewsl91.html#BrazdilGSTV91
  9. 9.
    Brazdil P, Muggleton S (2991) Learning to relate terms in a multiple agent environment. In: Kodratoff Y (ed) European Working Session on Learning (EWSL-91), Lecture Notes in Computer Science, vol 482, pp 424–439. Springer, Berlin. doi:10.1007/BFb0017035
  10. 10.
    Buşoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev 38(2):156–172CrossRefGoogle Scholar
  11. 11.
    Cao L (2001) Data mining and multi-agent integration. Springer, Boston. doi:10.1007/978-1-4419-0522-2
  12. 12.
    Cao L, Weiss G, Yu PS (2012) A brief introduction to agent mining. Auton Agents Multi Agent Syst 25(3):419–424CrossRefGoogle Scholar
  13. 13.
    Cao L, Luo C, Zhang C (2007) Agent-mining interaction: an emerging area. In: Gorodetsky V, Zhang C, Skormin VA, Cao L (eds) AIS-ADM, Lecture Notes in Computer Science, vol 4476, pp 60–73. Springer, Berlin. doi:10.1007/978-3-540-72839-9-5
  14. 14.
    Caragea D, Silvescu A, Honavar V (2001) Analysis and synthesis of agents that learn from distributed dynamic data sources. In: Wermter S, Austin J, Willshaw D (eds) Emergent neural computational architectures based on neuroscience, Lecture Notes in Computer Science, vol 2036. Springer, Berlin, pp 547–559. doi:10.1007/3-540-44597-8-39
  15. 15.
    Cetnarowicz K, Kisiel-Dorohinicki M, Nawarecki E (1996) The application of evolution process in multi-agent world to the prediction system. In: Tokoro M (ed) Proceedings of the International Conference on Multiagent Systems (ICMAS 96). AAAI Press, Menlo Park, pp 26–32Google Scholar
  16. 16.
    Chaimontree S, Atkinson K, Coenen F (2010) Clustering in a multi-agent data mining environment. In: Cao L, Bazzan AL, Gorodetsky V, Mitkas PA, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 5980, pp 103–114. Springer, Berlin. doi:10.1007/978-3-642-15420-1-9
  17. 17.
    Chaimontree S, Atkinson K, Coenen F (2012) A framework for multi-agent based clustering. Auton Agents Multi Agent Syst 25(3):425–446. doi:10.1007/s10458-011-9187-0 CrossRefGoogle Scholar
  18. 18.
    Chernova S, Veloso MM (2007) Confidence-based policy learning from demonstration using gaussian mixture models. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 1310–1317Google Scholar
  19. 19.
    da Silva JC, Giannella C, Bhargava R, Kargupta H, Klusch M (2005) Distributed data mining and agents. Eng Appl Artif Intell 18(7):791–807. doi:10.1016/j.engappai.2005.06.004 CrossRefGoogle Scholar
  20. 20.
    Davies W, Edwards P (1995) Distributed learning: an agent-based approach to data-mining. In: Proceedings of Machine Learning Workshop on Agents that Learn from Other AgentsGoogle Scholar
  21. 21.
    dos Santos DS, de Oliveira D, Bazzan ALC (2009) A multiagent, multiobjective clustering algorithm. In:Cao L (ed) Data mining and multiagent integration. Springer, Berlin. URL http://link.springer.com/chapter/10.1007
  22. 22.
    dos Santos DS, Bazzan ALC (2012) Distributed clustering for group formation and task allocation in multiagent systems: a swarm intelligence approach. Appl Soft Comput 12(8):2123–2131. doi:10.1016/j.asoc.2012.03.016 CrossRefGoogle Scholar
  23. 23.
    Emele C, Norman T, Şensoy M, Parsons S (2012) Learning strategies for task delegation in norm-governed environments. Auton Agents Multi Agent Syst 25(3):499–525. doi:10.1007/s10458-012-9194-9 CrossRefGoogle Scholar
  24. 24.
    Fiosins M, Fiosina J, Müller JP (2102) Change point analysis for intelligent agents in city traffic. In: Cao L, Bazzan AL, Symeonidis AL, Gorodetsky VI, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7103. Springer, Berlin, pp 195–210. doi:10.1007/978-3-642-27609-5-13
  25. 25.
    Garruzzo S, Rosaci D (2008) Agent clustering based on semantic negotiation. ACM Trans Auton Adapt Syst 3(2):7:1–7:40. doi:10.1145/1352789.1352792 CrossRefGoogle Scholar
  26. 26.
    Hindriks KV, Tykhonov D (2008) Opponent modelling in automated multi-issue negotiation using bayesian learning. In: Proceedings of the 7th Internatinal Joint Conference on Autonomous Agents and Multiagent Systems, pp 331–338Google Scholar
  27. 27.
    Kaelbling LP, Littman M, Moore A (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285Google Scholar
  28. 28.
    Kao Y, Cheng K (2006) An ACO-based clustering algorithm. In: Proceedings of the Fifth International Workshop on Ant Colony Optimization and Swarm Intelligence—ANTS 2006, vol 4150, Lecture Notes in Computer Science. Springer, Brussels, pp 340–347Google Scholar
  29. 29.
    Kargupta H, Hamzaoglu I, Stafford B, Stafford B (1997) Scalable, distributed data mining using an agent gased architecture. In: Proceedings of the Third International Conference on the Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, California, USA, pp 211–214Google Scholar
  30. 30.
    Kargupta H, Park B, Hershberger D, Johnson E (1999) Collective data mining: a new perspective toward distributed data mining. In: Advances in Distributed and Parallel Knowledge Discovery, vol 2. MIT Press, Cambridge, pp 131–174Google Scholar
  31. 31.
    Kiekintveld C, Miller J, Jordan PR, Wellman MP (2001) Forecasting market prices in a supply chain game. In: 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 1318–1325 (2007)Google Scholar
  32. 32.
    Klusch M, Lodi S, Moro G (2003) Issues of agent-based distributed data mining. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, New York, NY, USA, pp 1034–1035. doi:10.1145/860575.860782
  33. 33.
    Lemmens N, Tuyls K (2009) Stigmergic landmark foraging. In: Proceedings of the 8th International Joint Conference on Autonomous Agents and Multiagent Systems, pp 497–504Google Scholar
  34. 34.
    Lumer ED, Faieta B (1994) Diversity and adaptation in populations of clustering ants. In: Proceedings of the third international conference on Simulation of adaptive behavior: from animals to animats. MIT Press, Cambridge, MA, USA, pp 501–508Google Scholar
  35. 35.
    Mendoza MR, Bazzan ALC (submitted) Social choice in distributed classification tasks: dealing with vertically partitioned data. http://www.inf.ufrgs.br/maslab/pergamus/pubs/preprint_INS_bioinfovoting.pdf
  36. 36.
    Modi PJ, Shen WM (2001) Collaborative multiagent learning for classification tasks. In: Proceedings of the 5th International Conference on Autonomous Agents, AGENTS ’01. ACM, New York, NY, USA, pp 37–38. doi:10.1145/375735.375854
  37. 37.
    Nunes L, Oliveira EC (2004) Learning from multiple sources. In: Jennings N, Sierra C, Sonenberg L, Tambe M (eds) Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi Agent Systems, AAMAS, vol 3. IEEE Computer Society, New York, pp 1106–1113Google Scholar
  38. 38.
    Ogston E, Overeinder B, van Steen M, Brazier F (2003) A method for decentralized clustering in large multi-agent systems. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’03). ACM, New York pp 789–796. doi:10.1145/860575.860702
  39. 39.
    Palou XR, Rovatsos M (2009) Collaborative agent-based learning with limited data exchange. In: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS ’09). IFAAMAS, Richland, pp 1191–1192. URL http://dl.acm.org/citation.cfm?id=1558109.1558207
  40. 40.
    Pan R, Dolog P, Xu G (2013) KNN-based clustering for improving social recommender systems. In: Cao L, Zeng Y, Symeonidis AL, Gorodetsky VI, Yu PS , Singh MP (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7607. Springer, Berlin, pp 115–125. doi:10.1007/978-3-642-36288-0-11
  41. 41.
    Panait L, Luke S (2005) Cooperative multi-agent learning: the state of the art. Auton Agents Multi Agent Syst 11(3):387–434CrossRefGoogle Scholar
  42. 42.
    Peteiro-Barral D, Guijarro-Berdiñas B (2013) A survey of methods for distributed machine learning. Progress in AI 2(1):1–11Google Scholar
  43. 43.
    Piraveenan M, Prokopenko M, Wang P, Zeman A (2008) Decentralized multi-agent clustering in scale-free sensor networks. In: Fulcher J, Jain LC (eds) Computational intelligence: a compendium, studies in computational intelligence, vol 115. Springer, berlin, pp 485–515. doi:10.1007/978-3-540-78293-3-12
  44. 44.
    Provost FJ, Hennessy DN (1996) Scaling up: distributed machine learning with cooperation. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence. AAAI Press, Menlo park, pp 74–79Google Scholar
  45. 45.
    Provost F, Kolluri V (1999) A survey of methods for scaling up inductive algorithms. Data Min Knowl Discov 3(2):131–169CrossRefGoogle Scholar
  46. 46.
    Riley P, Veloso M (2000) On behavior classification in adversarial environments. In: Parker LE, Bekey G, Barhen J (eds) Distributed Autonomous Robotic Systems 4. Springer, Berlin, pp 371–380CrossRefGoogle Scholar
  47. 47.
    Riley P, Veloso M, Kaminka G (2002) An empirical study of coaching. In: Asama H, Arai T, Fukuda T, Hasegawa T (eds) Distributed Autonomous Robotic Systems 5. Springer, Berlin, pp 215–224CrossRefGoogle Scholar
  48. 48.
    Santana LEA, Canuto AMP, Xavier Jr. JC, Campos AMC: A comparative analysis of data distribution methods in an agent-based neural system for classification tasks. In: HIS, p 9Google Scholar
  49. 49.
    Santana LEO, Canuto AMP, Abreu MCC (2006) Analyzing the performance of an agent-based neural system for classification tasks using data distribution among the agents. In: IJCNN, pp 2951–2958Google Scholar
  50. 50.
    Şensoy M, Yilmaz B, Norman T (2013) Discovering frequent patterns to bootstrap trust. In: Cao L, Zeng Y, Symeonidis A, Gorodetsky V, Yu P, Singh MP (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol 7607, Springer, Berlin, pp 93–104. doi:10.1007/978-3-642-36288-0-9
  51. 51.
    Shelokar PS, Jayaraman VK, Kulkarni BD (2004) An ant colony approach for clustering. Anal Chim Acta 509(2):187–195CrossRefGoogle Scholar
  52. 52.
    Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365–377CrossRefMATHMathSciNetGoogle Scholar
  53. 53.
    Sian S (1991) Adaptation based on cooperative learning in multi-agent systems. In: Demazeau Y, Müller J (eds) Decentralized A.I., vol. 2. North-Holland, pp 257–272Google Scholar
  54. 54.
    Stolfo S, Tselepis ALPS, Prodromidis AL, Tselepis S, Lee W, Fan DW, Chan PK (1997) JAM: Java agents for meta-learning over distributed databases. In: Proceedings of 3rd International Confence on Knowledge Discovery and Data Mining. AAAI Press, Malno park, pp 74–81Google Scholar
  55. 55.
    Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective. Auton Robots 8(3):345–383CrossRefGoogle Scholar
  56. 56.
    Stone P (2007a) Learning and multiagent reasoning for autonomous agents. In: The 20th International Joint Conference on Artificial Intelligence, pp 13–30Google Scholar
  57. 57.
    Stone P (2007b) Multiagent learning is not the answer. It is the question. Artif Intell 171(7):402–405CrossRefMATHGoogle Scholar
  58. 58.
    Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  59. 59.
    Tozicka J, Rovatsos M, Pechoucek M (2007) A framework for agent-based distributed machine learning and data mining. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’07), ACM, New York, NY, USA, pp 666–673. doi:10.1145/1329125.1329243
  60. 60.
    Tuyls K, Weiss G (2012) Multiagent learning: basics, challenges, and prospects. AI Mag 33(3):41–52Google Scholar
  61. 61.
    Urban S, Jakob M, Pěchouček M (2010) Probabilistic modeling of mobile agents’ trajectories. In: Cao L, Bazzan AL, Gorodetsky V, Mitkas PA, Weiss G, Yu PS (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol. 5980, Springer, Berlin, pp 59–70. doi:10.1007/978-3-642-15420-1-6
  62. 62.
    Wardeh M, Coenen F, Bench-Capon T (2012) Multi-agent based classification using argumentation from experience. Auton Agents Multi Agent Syst 25(3):447–474. doi:10.1007/s10458-012-9197-6 CrossRefGoogle Scholar
  63. 63.
    Yang Y, Kamel M (2006) An aggregated clustering approach using multi-ant colonies algorithms. Pattern Recongn 39(7):1278–1289CrossRefMATHGoogle Scholar
  64. 64.
    Zhang W, Wang Y (2012) Agent-based cluster analysis of tropical cyclone tracks in the western north pacific. In: Cao L, Bazzan A, Symeonidis A, Gorodetsky V, Weiss G, Yu P (eds) Agents and data mining interaction, Lecture Notes in Computer Science, vol. 7103, Springer, Berlin, pp 98–113. doi:10.1007/978-3-642-27609-5-8

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Instituto de Informática, UFRGSPorto AlegreBrazil

Personalised recommendations