Skip to main content

Advertisement

SpringerLink
Log in

XCS-SL: a rule-based genetic learning system for sequence labeling

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Sequence labeling is an interesting classification domain where, like normal classification, every input has a class label, but unlike normal classification, prediction of an input’s label may depend on the values of other inputs or their classes, and so a learner may need to refer to inputs and classes at different time stamps to classify the current input. This is more difficult because a learner does not know where and how many inputs are needed to classify the current input. Our interest is in learning general rules for sequence labeling. The XCS algorithm is a rule-based knowledge discovery system powered by a genetic algorithm which has often been used for classification. Here we present XCS-SL, an extension of XCS classifier system which can be applicable to sequence labeling. Towards an application of Learning Classifier System (LCS) to sequence labeling, we propose a new classifier condition with memory (called a variable-length condition) and a rule-discovery system for the new classifier condition, which enables XCS to apply it to sequence labeling. In XCS-SL, classification rules (called “classifiers” here) can include extra conditions on previous inputs, which act as memories. In sequence labeling, the number of conditions/memories needed may be different for each input, hence, using a fixed number of conditions (i.e., fixed-length condition) for all classifiers is not a good solution. Instead, XCS-SL classifiers have a variable-length condition to provide more or less memory. The genetic algorithm can grow and shrink conditions to find a suitable memory size. On two synthetic benchmark problems XCS-SL learns optimal classifiers, and on a real-world sequence labeling task it derives high classification accuracy and discovers interesting knowledge that shows dependencies between inputs at different times. The comprehensively described system is the first application of a LCS to sequence labeling and we consider it a promising direction for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. In the algorithmic description [4], covering is activated when match set contains less than \(\theta _{mna}\) actions; however, \(\theta _{mna}\) is always set to the number of available actions so that the match set includes all the actions.

References

  1. Bache K, Lichman M (2013) UCI Machine Learning Repository. http://archive.ics.uci.edu/ml, University of California, Irvine, School of Information and Computer Sciences

  2. Bernadó-mansilla E, Garrell-Guij JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11:209–238

    Article  Google Scholar 

  3. Bui AAT, Taira RK (2009) Medical imaging informatics. Springer, Berlin

    Google Scholar 

  4. Butz MV, Wilson SW (2002) An algorithmic description of xcs. J Soft Comput 6(3–4):144–153

    Article  MATH  Google Scholar 

  5. Butz MV (2002) Anticipatory learning classifier systems, vol 4. Springer, Berlin

    MATH  Google Scholar 

  6. Carse B, Fogarty TC (1994) A delayed-action classifier system for learning in temporal environments. In: Evolutionary computation, 1994. Proceedings of the first IEEE conference on IEEE world congress on computational intelligence, vol 2. pp 670–673

  7. Chaovalitwongse WA, Prokopyev OA, Pardalos PM (2006) Electroencephalogram (EEG) time series classification: applications in epilepsy. Ann Oper Res 148(1):227–250

    Article  MATH  Google Scholar 

  8. Clark P, Niblett T (1989) The cn2 induction algorithm. In: Machine learning, pp 261–283

  9. Cliff D, Ross S (1994) Adding temporary memory to zcs. Adapt Behav 3(2):101–150

    Article  Google Scholar 

  10. Crammer Koby, Singer Yoram (2002) On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292

    MATH  Google Scholar 

  11. Dhanalakshmi V, Anand Kumar M, Rekha RU, Arun Kumar C, Soman KP, Rajendran S (2009) Morphological analyzer for agglutinative languages using machine learning approaches. In: Advances in recent technologies in communication and computing, 2009. ARTCom’09. International conference on IEEE, pp 433–435

  12. Dixon PW, Corne D, Oates MJ (2002) A preliminary investigation of modified xcs as a generic data mining tool. In: Revised papers from the 4th international workshop on advances in learning classifier systems, IWLCS ’01, London, UK, Springer, UK, pp 133–150

  13. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison Wesley, Boston

    MATH  Google Scholar 

  14. Grishman R, Sundheim B (1996) Message understanding conference-6: a brief history. In Proceedings of the 16th conference on computational linguistics—volume 1, COLING ’96, pp 466–471, Stroudsburg, PA, USA. Association for Computational Linguistics

  15. Han J (2005) Data mining: concepts and techniques. Morgan Kaufmann, San Francisco

    Google Scholar 

  16. Haykin S, Network N (2004) A comprehensive foundation. Neural Netw 2(2004)

  17. Holland JH (1986) Escaping brittleness: the possibilities of general purpose learning algorithms applied to parallel rule-based system. Mach Learn 2:593–623

    Google Scholar 

  18. Howard G, Bull L, Lanzi P-L (July 2010) A spiking neural representation for XCSF. In: Evolutionary computation (CEC), 2010 IEEE Congress on, pp 1–8

  19. Hurst J, Bull L (2004) A self-adaptive neural learning classifier system with constructivism for mobile robot control. In: Yao X, Burke EK, Lozano JA, Smith J, Merelo-Guervs JJ, Bullinaria JA, Rowe JE, Tio P, Kabn A, Schwefel H-P (eds) Parallel problem solving from nature—PPSN VIII, volume 3242 of Lecture Notes in Computer Science. Springer, Berlin, pp 942–951

    Google Scholar 

  20. Hurst J, Bull L, Melhuish C (2002) TCS learning classifier system controller on a real robot. In: Guervs JJM, Adamidis P, Beyer H-G, Schwefel H-P, Fernndez-Villacaas J-L (eds) Parallel problem solving from nature PPSN VII, volume 2439 of Lecture Notes in Computer Science. Springer, Berlin, pp 588–597

    Google Scholar 

  21. Kaluža B, Mirchevska V, Dovgan E, Luštrek M, Gams M (2010) An agent-based approach to care in independent living. In: Ambient intelligence, volume 6439 of LNCS Springer, Berlin, pp 177–186

  22. Kudo M, Toyama J, Shimbo M (1999) Multidimensional curve classification using passing-through regions. Pattern Recogn Lett 20(11–13):1103–1111

    Article  Google Scholar 

  23. Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: ICML2001, pp 282–289

  24. Lanzi PL (1999) Extending the representation of classifier conditions Part I: from binary to Messy coding. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference (GECCO-99), pp 337–344

  25. Lanzi PL, Wilson SW (2000) Toward optimal classifier system performance in non-markov environments. Evol Comput 8(4):393–418

    Article  Google Scholar 

  26. Nakata M, Kovacs T, Takadama K (2014) A modified XCS classifier system for sequence labeling. In: Proceedings of the 2014 conference on genetic and evolutionary computation, GECCO ’14, ACM, New York, NY, USA, pp 565–572

  27. Nguyen N, Guo Y (2007) Comparisons of sequence labeling algorithms and extensions. In: Proceedings of the 24th international conference on machine learning, ICML ’07, ACM, New York, NY, USA, pp 681–688

  28. Ordóñez FJ, de Toledo P, Sanchis A (2013) Activity recognition using hybrid generative/discriminative models on home environments using binary sensors. Sensors 13(5):5460–5477

    Article  Google Scholar 

  29. Peng F, McCallum A (2004) Accurate information extraction from research papers using conditional random fields. In: HLT-NAACL04, pp 329–336

  30. Pickering T (2013) Memory-based reinforcement learning in partially observable environments. Master’s thesis

  31. Preen R (2009) An xcs approach to forecasting financial time series. In: Proceedings of the 11th annual conference companion on genetic and evolutionary computation conference: Late Breaking Papers, GECCO ’09, ACM, New York, NY, USA, pp 2625–2632

  32. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106

    Google Scholar 

  33. Quinlan JR (1993) C4.5: Programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco

    Google Scholar 

  34. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2):257–286

    Article  Google Scholar 

  35. Roller BTCGD (2004) Max-margin markov networks. Adv Neural Inform Process Syst 16:25

    Google Scholar 

  36. Schmid H (1994) Probabilistic part-of-speech tagging using decision trees

  37. Stolzmann W (1998) Anticipatory classifier systems. Morgan Kaufmann, San Francisco

    Google Scholar 

  38. Stolzmann W (2000) An introduction to anticipatory classifier systems. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning Classifier Systems, volume 1813 of Lecture Notes in Computer Science. Springer, Berlin, pp 175–194

    Google Scholar 

  39. Tomlinson A, Bull L (2002) An accuracy based corporate classifier system. Soft Comput 6(3–4):200–215

    Article  MATH  Google Scholar 

  40. Tzima FA, Mitkas PA (2008) ZCS Revisited: Zeroth-level classifier systems for data mining. ICDM Workshops, pp 700–709

  41. Tzima FA, Mitkas PA (2013) Strength-based learning classifier systems revisited: effective rule evolution in supervised classification tasks. Eng Appl Artif Intel 26(2):818–832

    Article  Google Scholar 

  42. Vapnik VN, Vapnik V (1998) Statistical learning theory, vol 2. Wiley, New York

    MATH  Google Scholar 

  43. Webb A, Hart E, Ross P, Lawson A (2003) Controlling a simulated khepera with an xcs classifier system with memory. In: Banzhaf W, Ziegler J, Christaller T, Dittrich P, Kim JT (eds) Advances in artificial life, volume 2801 of Lecture Notes in Computer Science. Springer, Berlin, pp 885–892

    Google Scholar 

  44. Wilson SW (1994) ZCS: A Zeroth level classifier system. Evol Comput 2(1):1–18

    Article  Google Scholar 

  45. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  46. Wilson SW (1998) Generalization in the xcs classifier system. In: In Proceedings of 3rd the third annual genetic programming conference, pp 665–674

  47. Zhang K, Fan W (2008) Forecasting skewed biased stochastic ozone days: analyses, solutions and beyond. Knowl Inf Syst 14(3):299–326

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the JSPS Institutional Program for Young Researcher Overseas Visits.

Author information

Authors and Affiliations

  1. Graduate School of Informatics, The University of Electro-Communications, Chofu, 182-0021, Japan

    Masaya Nakata & Keiki Takadama

  2. Japan Society for the Promotion of Science, Chiyoda-ku, 102-0083, Japan

    Masaya Nakata

  3. Department of Computer Science, University of Bristol, Bristol, BS8 1UB, UK

    Tim Kovacs

Authors
  1. Masaya Nakata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Kovacs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keiki Takadama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaya Nakata.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakata, M., Kovacs, T. & Takadama, K. XCS-SL: a rule-based genetic learning system for sequence labeling. Evol. Intel. 8, 133–148 (2015). https://doi.org/10.1007/s12065-015-0127-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-015-0127-9

Keywords

Search

Navigation