Advertisement

On ZCS in multi-agent environments

  • Larry Bull
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1498)

Abstract

This paper examines the performance of the ZCS Michigan-style classifier system in multi-agent environments. Using an abstract multi-agent model the effects of varying aspects of the performance, reinforcement and discovery components are examined. It is shown that small modifications to the basic ZCS architecture can improve its performance in environments with significant inter-agent dependence. Further, it is suggested that classifier systems have characteristics which make them more suitable to such non-stationary problem domains in comparison to other forms of reinforcement learning. Results from the initial use of ZCS as an adaptive economic trading agent within an artificial double-auction market are then presented, with the findings from the abstract model shown to improve the efficiency of the traders and hence the overall market.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arthur W B (1990), “A Learning Algorithm that Replicates Human Learning”, Technical Report 90-026, Santa Fe Institute.Google Scholar
  2. 2.
    Axlerod R (1987), “The Evolution of Strategies in the Iterated Prisoner's Dilemma,” in L Davis (ed.) Genetic Algorithms and Simulated Annealing, Pittman, pp32–42.Google Scholar
  3. 3.
    Booker L (1985), “Improving the Performance of Genetic Algorithms in Classifier Systems”, in J J Grefenstette (ed.) Proceedings of the First International Conference on Genetic Algorithms and their Applications, Lawrence Erlbaum, pp80–93.Google Scholar
  4. 4.
    Booker L (1989), “Triggered Rule Discovery in Classifier Systems”, in J D Schaffer (ed.) Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, pp265–275.Google Scholar
  5. 5.
    Bull L (1998), “Evolutionary Computing in Multi-Agent Environments: Operators,” in V W Porto, N Saravanan, D E Waagen & A E Eiben (eds.) Proceedings of the Seventh Annual Conference on Evolutionary Programming, Springer Verlag, to appear.Google Scholar
  6. 6.
    Bull L (1997), “Evolutionary Computing in Multi-Agent Environments: Partners,” in T Baeck (ed.) Proceedings of the Seventh International Conference on Genetic Algorithms, Morgan Kaufmann, pp370–377.Google Scholar
  7. 7.
    Bull L, Fogarty T C & Snaith M (1995), “Evolution in Multi-Agent Systems: Evolving Communicating Classifier Systems for Gait in a Quadrupedal Robot,” in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp382–388.Google Scholar
  8. 8.
    Carse B, Fogarty T C & Munro A (1995), “Adaptive Distributed Routing using Evolutionary Fuzzy Control”, in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp389–397.Google Scholar
  9. 9.
    Cedeno W & Vemuri V (1997), “On the Use of Niching for Dynamic Landscapes”, in Proceedings of the 1997 IEEE International Conference on Evolutionary Computation, IEEE, pp361–366.Google Scholar
  10. 10.
    Cliff D & Bruten J (1997), “Zero is Not Enough: On the Lower Limit of Agent Intelligence for Continuous Double Auction Markets”, HP Laboratories Technical Report HPL-97-141, HP Laboratories Bristol.Google Scholar
  11. 11.
    Dorigo M & Schnepf U (1992), “Genetics-based Machine Learning and Behaviour-based Robotics: A New Synthesis”, IEEE Trans. on Sys. Man and Cybernetics 22(6):141–154.Google Scholar
  12. 12.
    Dorigo M & Bersini H (1994), “A Comparison of Q-learning and Classifier Systems”, in D Cliff, P Husbands, J-A Meyer & S W Wilson (eds.) From Animals to Animats 3, MIT Press, pp248–255.Google Scholar
  13. 13.
    Dworman G (1994), “Games Computers Play: Simulating Characteristic Function Game Playing Agents with Classifier Systems”, in Proceedings of the 1994 IEEE Conference on Evolutionary Computing, IEEE.Google Scholar
  14. 14.
    Holland J H (ed.)(1975), Adaptation in Natural and Artificial Systems, University of Michigan Press.Google Scholar
  15. 15.
    Holland J H, Holyoak K J, Nisbett R E & Thagard P R (eds.)(1986), Induction: Processes of Inference, Learning and Discovery, MIT Press.Google Scholar
  16. 16.
    Kauffman S A (ed.)(1993), The Origins of Order: Self-organisation and Selection in Evolution, Oxford University Press.Google Scholar
  17. 17.
    Lin L-J (1992), “Self-improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching”, Machine Learning 8(3):293–322.Google Scholar
  18. 18.
    Marengo L & Tordjman H (1996), “Speculation, Heterogeneity and Learning: A Model of Exchange Rate Dynamics” KYKLOS 49(3):407–438.Google Scholar
  19. 19.
    Marimon R, McGrattan E & Sargent T (1990), “Money as a Medium of Exchange in an Economy with Artificially Intelligent Agents”, Economic Dynamics and Control (14):329–373.MATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    Mitlohner J (1996), “Classifier Systems and Economic Modelling” APL Quote Quad 26(4)Google Scholar
  21. 21.
    Palmer R, Arthur W B, Holland J H, LeBaron B & Tayler P (1994), “Artificial Economic Life: A Simple Model of a Stockmarket”, Physica D 75:264–274.MATHCrossRefGoogle Scholar
  22. 22.
    Potter M, De Jong K & Grefenstette J (1995), “A Coevolutionary Approach to Learning Sequential Decision Rules”, in L J Eshelman (ed.) Proceedings of the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp366–372.Google Scholar
  23. 23.
    Sandholm T & Crites R H (1995), “Multiagent Reinforcement Learning in the Iterated Prisoner's Dilemma”, BioSystems 37: 147–166.CrossRefGoogle Scholar
  24. 24.
    Seredynski F, Cichosz P & Klebus G (1995), “Learning Classifier Systems in Multi-Agent Environments”, in Proceedings of the First IEE/IEEE Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, IEE, pp287–292.Google Scholar
  25. 25.
    Smith S F (1980), “A Learning System Based on Genetic Adaptive Algorithms”, PhD dissertation, University of Pittsburgh.Google Scholar
  26. 26.
    Smith V (ed.)(1992), Papers in Experimental Economics, Cambridge Press.Google Scholar
  27. 27.
    Syswerda G (1989), “Uniform Crossover in Genetic Algorithms”, in J D Schaffer (ed.) Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, pp2–9.Google Scholar
  28. 28.
    Tsetlin M (ed.)(1973), Automaton Theory and Modeling of Biological Systems, Academic Press.Google Scholar
  29. 29.
    Watkins C (1989), “Learning from Delayed Rewards”, PhD dissertation, University of Cambridge.Google Scholar
  30. 30.
    Weiss G (ed.)(1997), Distributed Artificial Intelligence Meets Machine Learning, Springer.Google Scholar
  31. 31.
    Wilson S W (1994), “ZCS: A Zeroth-level Classifier System”, Evolutionary Computation 2(1):1–18.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Larry Bull
    • 1
  1. 1.Intelligent Computer Systems CentreUniversity of the West of EnglandBristolUK

Personalised recommendations