Abstract
Extended classifier system (XCS) is an extension of a popular online rule-based machine learning technique, learning classifier system (LCS), in which a classifier’s fitness is based on its accuracy instead of the prediction itself, and a genetic algorithm (GA) and reinforcement learning (RL) component is utilized for exploratory and learning purposes, respectively. With the emergence of increasingly intricate rule-based learning techniques, there is a need to examine feasible methods of learning that can overcome the challenges posed by complex scenarios while supporting online performance. Checkers is a strategic, combinatorial game having a high branching factor and a complex state space that provides a promising avenue for scrutinizing novel approaches. This paper presents a preliminary investigation into feasibility of XCS in such complex avenues by taking 6 × 6 checkers as a specific case of study. The XCS agent was adapted to this problem, trained with random agent and was able to perform well against the alpha–beta pruning algorithm of various depths as well as human agents of different skill levels (beginner, intermediate and advanced).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bull L (2004) Learning classifier systems: a brief introduction. In: Applications of learning classifier systems, pp 1–12
Snell F (1967) Progress in theoretical biology. Academic Press, New York
Holland J (2001) Adaptation in natural and artificial systems. MIT Press, Cambridge
Holland J, Holyoak K, Nisbett R, Thagard P, Smoliar S (1987) Induction: processes of inference, learning, and discovery. IEEE Expert 2:92–93
Wilson S (1995) Classifier fitness based on accuracy. Evol Comput 3:149–175
Silver D, Huang A, Maddison C, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489
Silver D, Hubert T, Schrittwieser J, Antonoglou I, Lai M, Guez A, Lanctot M, Sifre L, Kumaran D, Graepel T, Lillicrap T, Simonyan K, Hassabis D (2018) A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362:1140–1144
Dubel ICL, Lefakis L (2006) Checkers reinforcement learning project: AI checkers player
Kwasnicka H, Spirydowicz A (2019) Checkers: TD (λ) learning applied for deterministic game
Tesauro G (1995) Temporal difference learning and TD-Gammon. Commun ACM 38:58–68
Eck N, Wezel M (2005) Reinforcement learning and its application to Othello
Van der Ree M, Wiering M (2013) Reinforcement learning in the game of Othello: learning against a fixed opponent and learning from self-play. In: 2013 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL)
Knuth D, Moore R (1975) An analysis of alpha-beta pruning. Artif Intell 6:293–326
Castillo C, Lurgi M, Martinez I (2003) Chimps: an evolutionary reinforcement learning approach for soccer agents. In: SMC’03 conference proceedings. 2003 IEEE international conference on systems, man and cybernetics. Conference theme—system security and assurance (Cat. No. 03CH37483)
Rudolph S, von Mammen S, Jungbluth J, Hähner J (2016) Design and evaluation of an extended learning classifier-based StarCraft micro AI. In: Applications of evolutionary computation, pp 669–681
Shafi K, Abbass H (2017) A survey of learning classifier systems in games [review article]. IEEE Comput Intell Mag 12:42–55
Schaeffer J, Culberson J, Treloar N, Knight B, Lu P, Szafron D (1992) A world championship caliber checkers program. Artif Intell 53:273–289
Schaeffer J, Burch N, Bjornsson Y, Kishimoto A, Muller M, Lake R, Lu P, Sutphen S (2007) Checkers is solved. Science 317:1518–1522
Browne W, Scott D (2005) An abstraction algorithm for genetics-based reinforcement learning. In: Proceedings of the 2005 conference on genetic and evolutionary computation—GECCO’05
Jain S, Verma S, Kumar S, Aggarwal S (2018) An evolutionary learning approach to play Othello using XCS. In: 2018 IEEE congress on evolutionary computation (CEC)
Clementis L (2013) Model driven classifier evaluation in rule-based system. In: Advances in intelligent systems and computing, pp 267–276
Urbanowicz R, Moore J (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1–25
Butz M, Wilson S (2001) An algorithmic description of XCS. In: Advances in learning classifier systems, pp 253–272
Wilson S (2000) Get real! XCS with continuous-valued inputs. In: Lecture notes in computer science, pp 209–219
Browne W (2004) The development of an industrial learning classifier system for data-mining in a steel hop strip mill. In: Applications of learning classifier systems, pp 223–259
Lanzi P (2002) Learning classifier systems from a reinforcement learning perspective. Soft Comput 6:162–170
Zhong J, Hu X, Zhang J, Gu M (2005) Comparison of performance between different selection strategies on simple genetic algorithms. In: International conference on computational intelligence for modelling, control and automation and international conference on intelligent agents, web technologies and internet commerce (CIMCA-IAWTIC’06)
Shuqin L, Weiming X, Xiaohua Y (2015) Study on the evaluation function parameters of the checkers game program on Weka platform. Int J New Technol Res 1(7)
Su Z, Li S, Jia Y, Zheng L, Fan S (2013) The realization of genetic algorithm in terms of checkers evaluation function. Appl Mech Mater 411–414:1979–1985
Tang K, Jarvis R (2005) Is XCS suitable for problems with temporal rewards? In: International conference on computational intelligence for modelling, control and automation and international conference on intelligent agents, web technologies and internet commerce (CIMCA-IAWTIC’06)
Barry A (2003) Limits in long path learning with XCS. In: Genetic and evolutionary computation—GECCO 2003, pp 1832–1843
Lanzi P, Loiacono D (2007) Classifier systems that compute action mappings. In: Proceedings of the 9th annual conference on genetic and evolutionary computation—GECCO’07
Butz M, Goldberg D, Lanzi P (2004) Gradient-based learning updates improve XCS performance in multistep problems. In: Genetic and evolutionary computation—GECCO 2004, pp 751–762
Hiniker A, Daniels J, Williamson H (2013) Go go games. In: Proceedings of the 12th international conference on interaction design and children—IDC’13
Li X, Yang G (2016) Transferable XCS. In: Proceedings of the 2016 on genetic and evolutionary computation conference—GECCO’16
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Oberoi, K., Tandon, S., Das, A., Aggarwal, S. (2021). An Evolutionary Approach to Combinatorial Gameplaying Using Extended Classifier Systems. In: Choudhary, A., Agrawal, A.P., Logeswaran, R., Unhelkar, B. (eds) Applications of Artificial Intelligence and Machine Learning. Lecture Notes in Electrical Engineering, vol 778. Springer, Singapore. https://doi.org/10.1007/978-981-16-3067-5_54
Download citation
DOI: https://doi.org/10.1007/978-981-16-3067-5_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-3066-8
Online ISBN: 978-981-16-3067-5
eBook Packages: Computer ScienceComputer Science (R0)