Abstract
The exploration–exploitation trade-off has been one of the most common problems in reinforcement learning. There have been multiple policies in the past that have tried to solve this issue optimally. We propose a redesigned version of the classical multi-arm bandit problem. The new environment formulates the multi-arm bandit problem as an episodic task with the possibility of termination in the middle of the episode. This task tests the ability of the agent to explore the environment as the states change significantly while proceeding through it. We also propose a policy- —segmented \(\varepsilon \)-Greedy—that allows the agent to pass through the environment while maximizing its returns along the way. This policy has been compared with existing policies on our proposed environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Biewald L (2020) Experiment tracking with weights and biases. https://www.wandb.com/, software available from wandb.com
Gupta AK, Smith KG, Shalley CE (2006) The interplay between exploration and exploitation. Acad Manage J 49(4):693–706
Hunter JD (2007) Matplotlib: a 2d graphics environment. Comput Sci Eng 9(3):90–95
Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285
March JG (1991) Exploration and exploitation in organizational learning. Organization Sci 2(1):71–87
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602
Rao R, Narasimhan K (2020) m-stage epsilon-greedy exploration for reinforcement learning
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. A Bradford Book, Cambridge, MA, USA, pp 32–33
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. A Bradford Book, Cambridge, MA, USA, pp 34–36
Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 30
Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. In: International conference on machine learning. PMLR, pp 1995–2003
Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Shankar, A., Diwan, M., Marathe, A., Takalikar, M. (2023). Segmented \(\varepsilon \)-Greedy for Solving a Redesigned Multi-arm Bandit Environment. In: Das, S., Saha, S., Coello Coello, C.A., Bansal, J.C. (eds) Advances in Data-Driven Computing and Intelligent Systems. ADCIS 2022. Lecture Notes in Networks and Systems, vol 698. Springer, Singapore. https://doi.org/10.1007/978-981-99-3250-4_22
Download citation
DOI: https://doi.org/10.1007/978-981-99-3250-4_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-3249-8
Online ISBN: 978-981-99-3250-4
eBook Packages: EngineeringEngineering (R0)