Abstract
Learning agents have to deal with the exploration-exploitation dilemma. The choice between exploration and exploitation is very difficult in dynamic systems; in particular in large scale ones such as economic systems. Recent research shows that there is neither an optimal nor a unique solution for this problem. In this paper, we propose an adaptive approach based on meta-rules to adapt the choice between exploration and exploitation. This new adaptive approach relies on the variations of the performance of the agents. To validate the approach, we apply it to economic systems and compare it to two adaptive methods: one local and one global. Herein, we adapt these two methods, which were originally proposed by Wilson, to economic systems. Moreover, we compare different exploration strategies and focus on their influence on the performance of the agents.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Azoulay-Schwartz, R., Kraus, S., Wilkenfeld, J.: Exploration vs. exploitation: choosing a supplier in an environment of incomplete information. Elsevier Science, Amsterdam (2003)
Baum, J.A.C., Rao, H.: Handbook of Organizational Change and Development: Evolutionary Dynamics of Organizational Populations and Communities. Oxford University Press, Oxford (1999)
Butz, M.V., Wilson, S.W.: An algorithmic description of XCS. Journal of Soft Computing 6, 144–153 (2002)
Carmel, D., Markovitch, S.: Exploration Strategies for Model-Based Learning in Multi-agent Systems. In: Jennings, N., Sycara, K., Georgeff, M. (eds.) Autonomous Agents and Multi-agent systems, vol. 2(2), pp. 141–172 (1999)
Gittings, J.C.: Multi-armed bandit allocation indices. John Wiley and Sons, NY (1989)
Kaelbling, L.P., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Meuleau, N., Bourgine, P.: Exploration of multi-state environments: Local measure and back-propagation of uncertainty. Machine Learning 35(2), 117–154 (1999)
Miramontes Hercog, L., Fogarty, T.C.: Social Simulation Using a Multi-agent Model Based on Classifier Systems: The emergence of Vacillating Behavior in the ”El Farol” Bar Problem. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2001. LNCS (LNAI), vol. 2321, pp. 88–111. Springer, Heidelberg (2002)
Rejeb, L., Guessoum, Z.: Adaptive Firms. In: Proc. AISTA 2004 International Conference on Advances in Intelligent Systems - Theory and Applications. IEEE Computer Society, Luxembourg (2004)
Penrose, E.T.: The theory of the growth of the firm. Basil Blackwell, Malden (1959)
Peres-Uribe, A., Hirsbrunner, B.: The risk of Exploration in multi-agent learning systems: a case study. In: Proc. Agents 2000 Joint workshop on learning agents, Barcelona, June 3–7, pp. 33–37 (2000)
Sutton, R.S., Barto, A.G.: Reinforcement learning, an introduction. The MIT Press, Cambridge (1998)
Thrun, S.B.: The role of exploration in learning control. In: Sofge, D.A. (ed.) Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches, Van Nostrand Reinhold, Florence (1992)
Watkins, C., Dayan, P.: Q-Learning. Machine Learning 8, 279–292 (1999)
Wiering, M.: Explorations in Efficient Reinforcement Learning. Ph.D. thesis (February 1999)
Wilson, S.W.: Classifiers Fitness Based on Accuracy. Evolutionary computation 3(2), 149–175 (1995)
Wilson, S.W.: Explore/Exploit Strategies in Autonomy. In: Maes, P., Mataric, M., Pollac, J., Meyer, J.-A., Wilson, S. (eds.) From Animals to Animats 4, Proc. of the 4th International Conference of Adaptive Behavior, Cambridge (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rejeb, L., Guessoum, Z., M’Hallah, R. (2005). An Adaptive Approach for the Exploration-Exploitation Dilemma for Learning Agents. In: Pěchouček, M., Petta, P., Varga, L.Z. (eds) Multi-Agent Systems and Applications IV. CEEMAS 2005. Lecture Notes in Computer Science(), vol 3690. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11559221_32
Download citation
DOI: https://doi.org/10.1007/11559221_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29046-9
Online ISBN: 978-3-540-31731-9
eBook Packages: Computer ScienceComputer Science (R0)