Soft Computing

, Volume 17, Issue 6, pp 953–981 | Cite as

GAssist vs. BioHEL: critical assessment of two paradigms of genetics-based machine learning

  • María A. Franco
  • Natalio Krasnogor
  • Jaume Bacardit
Focus

Abstract

This paper reports an exhaustive analysis performed over two specific Genetics-based Machine Learning systems: BioHEL and GAssist. These two systems share many mechanisms and operators, but at the same time, they apply two different learning paradigms (the Iterative Rule Learning approach and the Pittsburgh approach, respectively). The aim of this paper is to: (a) propose standard configurations for handling small and large datasets, (b) compare the two systems in terms of learning capabilities, complexity of the obtained solutions and learning time, (c) determine the areas of the problem space where each one of these two systems performs better, and (d) compare them with other well-known machine learning algorithms. The results show that it is possible to find standard configurations for both systems. With these configurations the systems perform up to the standards of other state-of-the-art machine learning algorithms such as Support Vector Machines. Moreover, we identify the problem domains where each one of these systems have advantages and disadvantages and propose ways to improve the systems based on this analysis.

Keywords

Genetics-based machine learning Classification Data mining Comparison methodologies Sensitivity and scalability analysis 

Supplementary material

500_2013_1016_MOESM1_ESM.pdf (120 kb)
PDF (121 KB)

References

  1. Aguilar-Ruiz J, Riquelme J, Toro M (2003) Evolutionary learning of hierarchical decision rules. IEEE Trans Syst Man Cybern Part B 33(2):324–331CrossRefGoogle Scholar
  2. Aha DW, Kibler D, Albert MK (1991) Instance based learning algorithms. Mach Learn 6:37–66Google Scholar
  3. Alcalá-Fdez J, Sánchez L, García S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernández JC, Herrera F (2009) Keel: a software tool to assess evolutionary algorithms for data mining problems. Soft Comput 13:307–318CrossRefGoogle Scholar
  4. Bacardit J (2004) Pittsburgh Genetics-Based machine learning in the data mining era: Representations, generalization, and run-time. PhD thesis, Ramon Llull University, Barcelona, SpainGoogle Scholar
  5. Bacardit J, Butz M (2007) Data mining in learning classifier systems: Comparing XCS with GAssist. In: Kovacs T, Llorà à X, Takadama K, Lanzi P, Stolzmann W, Wilson S (eds) Learning classifier systems. Lecture Notes in computer science, vol 4399. Springer, Berlin, pp 282–290Google Scholar
  6. Bacardit J, Garrell JM (2003a) Bloat control and generalization pressure using the minimum description length principle for a pittsburgh approach learning classifier system. In: Proceedings of the 6th international workshop on learning classifier systemsGoogle Scholar
  7. Bacardit J, Garrell JM (2003b) Evolving multiple discretizations with adaptive intervals for a pittsburgh Rule-Based learning classifier system. In: Proceedings of the genetic and evolutionary computation conference-GECCO2003, LNCS 2724, Springer, Berlin, pp 1818–1831Google Scholar
  8. Bacardit J, Krasnogor N (2008) Empirical evaluation of ensemble techniques for a pittsburgh learning classifier system. In: Learning classifier systems. Lecture notes on computer science, vol 4998. Springer, Berlin, pp 255–268Google Scholar
  9. Bacardit J, Krasnogor N (2009a) A mixed discrete-continuous attribute list representation for large scale classification domains. In: GECCO ’09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation. ACM Press, New York, pp 1155–1162Google Scholar
  10. Bacardit J, Krasnogor N (2009b) Performance and efficiency of memetic pittsburgh learning classifier systems. Evolut Comput J 17(3)Google Scholar
  11. Bacardit J, Goldberg DE, Butz MV, Llorá X, Garrell JM (2004) Speeding-up Pittsburgh learning classifier systems: modeling time and accuracy. In: Parallel Problem Solving from Nature-PPSN VIII. Lecture notes in computer science, vol 3242, chap 103. Springer, Berlin, pp 1021–1031Google Scholar
  12. Bacardit J, Bernadó-Mansilla E, Butz MV (2007a) Learning classifier systems: Looking back and glimpsing ahead. In: Bacardit J, Bernadó-Mansilla E, Butz MV, Kovacs T, Llorà à X, Takadama K (eds) IWLCS, Lecture Notes in Computer Science, vol 4998, Springer, Berlin, pp 1–21Google Scholar
  13. Bacardit J, Goldberg DE, Butz MV (2007b) Improving the performance of a Pittsburgh learning classifier system using a default rule. In: Learning Classifier Systems, Revised Selected Papers of the International Workshop on Learning Classifier Systems 2003–2005, LNCS 4399, Springer, Berlin, pp 291–307Google Scholar
  14. Bacardit J, Burke EK, Krasnogor N (2009a) Improving the scalability of rule-based evolutionary learning. Memetic Comput 1(1):55–67CrossRefGoogle Scholar
  15. Bacardit J, Stout M, Hirst JD, Valencia A, Smith R, Krasnogor N (2009b) Automated alphabet reduction for protein datasets. BMC Bioinform 10(1):6CrossRefGoogle Scholar
  16. Bassel GW, Glaab E, Marquez J, Holdsworth MJ, Bacardit J (2011) Functional network construction in arabidopsis using rule-based machine learning on large-scale data sets. Plant Cell Online 23(9):3101–3116CrossRefGoogle Scholar
  17. Bernadó-Mansilla E, Garrell JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238CrossRefGoogle Scholar
  18. Bernadó-Mansilla E, Llorà à X, Garrell JM (2006) XCS and GALE: a comparative study of two learning classifier systems on data mining. In: Lanzi P, Stolzmann W, Wilson S (eds) Advances in learning classifier systems. Lecture notes in computer science, chap 8, vol 2321. Springer, Berlin, pp 115–132Google Scholar
  19. Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. url:(http://www.ics.uci.edu/mlearn/MLRepository.html)
  20. Browne WN, Ioannides C (2007) Investigating scaling of an abstracted LCS utilising ternary and s-expression alphabets. In: Proceedings of the 2007 GECCO conference companion on genetic and evolutionary computation. ACM Press, London, pp 2759–2764Google Scholar
  21. Bull L (2001) Simple markov models of the genetic algorithm in classifier systems: Multi-step tasks. In: IWLCS ’00: revised papers from the third international workshop on advances in learning classifier systems. Springer, London, pp 29–36Google Scholar
  22. Bull L, Hurst J (2000) Self-Adaptive mutation in ZCS controllers. In: Lecture notes in computer science, chapter 33, vol 1803. Springer, Berlin, pp 342–349Google Scholar
  23. Bull L, Studley M, Bagnall A, Whittley I (2007) Learning classifier system ensembles with rule-sharing. IEEE Trans Evolut Comput 11(4):496–502CrossRefGoogle Scholar
  24. Butz MV (2005) Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. In: Proceedings genetic evolutionary computation conference GECCO 2005. ACM, New York, pp 1835–1842Google Scholar
  25. Butz MV, Herbort O (2008) Context-dependent predictions and cognitive arm control with XCSF. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, GECCO ’08. ACM Press, New York, pp 1357–1364Google Scholar
  26. Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems. IEEE Trans Evolut Comput 9(5):452–473CrossRefGoogle Scholar
  27. Butz MV, Lanzi PL, Llorà à X, Loiacono D (2008a) An analysis of matching in learning classifier systems. In: GECCO ’08: Proceedings of the 10th annual conference on genetic and evolutionary computation. ACM Press, New York, pp 1349–1356Google Scholar
  28. Butz MV, Lanzi PL, Wilson SW (2008b) Function approximation with XCS: hyperellipsoidal conditions, recursive least squares, and compaction. IEEE Trans Evolut Comput 12(3):355–376CrossRefGoogle Scholar
  29. Butz MV, Stalph PO, Lanzi PL (2008c) Self-adaptive mutation in XCSF. In: Proceedings of the 10th annual conference on genetic and evolutionary computation. ACM Press, Atlanta, pp 1365–1372Google Scholar
  30. De Jong K (1988) Learning with genetic algorithms: an overview. Mach Learn 3(2-3):121–138CrossRefGoogle Scholar
  31. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATHGoogle Scholar
  32. Franco M, Martínez I, Gorrin C (2010a) Supply chain management sales using XCSR. In: Bacardit J, Browne W, Drugowitsch J, Bernadó-Mansilla E, Butz M (eds) Learning classifier systems. Lecture notes in computer science, vol 6471, Springer, Berlin, pp 145–165Google Scholar
  33. Franco MA, Krasnogor N, Bacardit J (2010b) Analysing BioHEL using challenging boolean functions. In: GECCO ’10: Proceedings of the 12th annual conference comp on genetic and evolutionary computation. ACM Press, New York, pp 1855–1862Google Scholar
  34. Franco MA, Krasnogor N, Bacardit J (2010c) Speeding up the evaluation of evolutionary learning systems using GPGPUs. In: GECCO ’10: Proceedings of the 12th annual conference on genetic and evolutionary computation. ACM, New York, pp 1039–1046Google Scholar
  35. Frank E, Witten IH (1998) Generating accurate rule sets without global optimization. In: Proceedings of the fifteenth international conference on machine learning, ICML ’98. Morgan Kaufmann Publishers Inc., San Francisco, pp 144–151Google Scholar
  36. Freitas AA (2002) Data mining and knowledge discovery with evolutionary algorithms. Springer, New YorkGoogle Scholar
  37. Freitas AA (2008) A review of evolutionary algorithms for data mining. In: Maimon O, Rokach L (eds) Soft computing for knowledge discovery and data mining. Springer US, pp 79–111Google Scholar
  38. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701CrossRefGoogle Scholar
  39. García S, Fernández A, Luengo J, Herrera F (2009) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput Fusion Found Methodol Appl 13(10):959–977Google Scholar
  40. Goldberg DE (1989) Genetic algorithms for search, optimization and machine learning. Addison-Wesley Longman Publishing Co., Inc., BostonGoogle Scholar
  41. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. SIGKDD Explor Newsl 11(1):10–18CrossRefGoogle Scholar
  42. Holland J (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Ann ArborGoogle Scholar
  43. Holland JH, Reitman JS (1978) Cognitive systems based on adaptive algorithms. In: Hayes-Roth D, Waterman F (eds) Pattern-directed inference systems. Academic Press, New York, pp 313–329Google Scholar
  44. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Statist 6(2):65–70, ArticleType: primary_article / Full publication date: 1979 / Copyright 1979 Board of the Foundation of the Scandinavian Journal of StatisticsGoogle Scholar
  45. Hruschka E, Campello R, Freitas A, de Carvalho A (2009) A survey of evolutionary algorithms for clustering. IEEE Trans Syst Man Cybern C 39(2):133–155CrossRefGoogle Scholar
  46. Hurst J, Bull L (2001) Self-Adaptation in classifier system controllers. Artif Life Robotics 5:109–119CrossRefGoogle Scholar
  47. Hurst J, Bull L (2002) A self-adaptive XCS. In: Lanzi P, Stolzmann W, Wilson S (eds) Advances in learning classifier systems. Lecture notes in computer science, vol 2321, Springer, Berlin, pp 333–360. doi:i0.1007/3-540-48104-4_5
  48. Hurst J, Bull L (2006) A neural learning classifier system with self-adaptive constructivism for mobile robot control. Artif Life 12:353–380CrossRefGoogle Scholar
  49. Janikow CZ (1993) A knowledge-intensive genetic algorithm for supervised learning. Mach Learn 13(2-3):189–228CrossRefGoogle Scholar
  50. Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12CrossRefGoogle Scholar
  51. John G, Langley P (1995) Estimating continuous distributions in bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence. Morgan Kaufmann, Burlington, pp 338–345Google Scholar
  52. Jong KD, Spears WM (1991) Learning concept classification rules using genetic algorithms. In: Proceedings of the 12th international joint conference on artificial intelligence, vol 2, Morgan Kaufmann Publishers Inc., Sydney, pp 651–656Google Scholar
  53. Lanzi PL (2008) Learning classifier systems: then and now. Evolut Intell 1(1):63–82CrossRefGoogle Scholar
  54. Lanzi PL, Perrucci A (1999a) Extending the representation of classifier conditions part I: from binary to messy coding. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference, vol 1. Morgan Kaufmann, Orlando, pp 345–352Google Scholar
  55. Lanzi PL, Perrucci A (1999b) Extending the representation of classifier conditions part II: from messy coding to S-Expressions. In: Banzhaf W, Daida J, Eiben AE, Garzon MH, Honavar V, Jakiela M, Smith RE (eds) Proceedings of the genetic and evolutionary computation conference, vol 1. Morgan Kaufmann, Orlando, pp 345–352Google Scholar
  56. Lanzi PL, Wilson SW (2006) Using convex hulls to represent classifier conditions. In: GECCO ’06: Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM Press, New York, pp 1481–1488Google Scholar
  57. Llorà X, Garrell JM (2000) Evolving agent aggregates using cellular genetic algorithms. In: Whitley LD, Goldberg DE, Cantú-Paz E, Spector L, Parmee IC, Beyer HG (eds) GECCO. Morgan Kaufmann, Burlington, p 868Google Scholar
  58. Llorà X, Sastry K (2006) Fast rule matching for learning classifier systems via vector instructions. In: GECCO ’06: Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM Press, New York, pp 1513–1520Google Scholar
  59. Llorà X, Reddy R, Matesic B, Bhargava R (2007a) Towards better than human capability in diagnosing prostate cancer using infrared spectroscopic imaging. In: Proceedings of the 9th annual conference on genetic and evolutionary computation, GECCO ’07. ACM Press, New York, pp 2098–2105Google Scholar
  60. Llorà X, Sastry K, Yu T, Goldberg DE (2007b) Do not match, inherit: fitness surrogates for genetics-based machine learning techniques. In: GECCO ’07: Proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, New York, pp 1798–1805Google Scholar
  61. Mellor D (2005) A first order logic classifier system. In: GECCO ’05: Proceedings of the 2005 conference on genetic and evolutionary computation. ACM Press, New York, pp 1819–1826Google Scholar
  62. Nemenyi P (1963) Distribution-free multiple comparisons. PhD thesis, Princeton University, USAGoogle Scholar
  63. Orriols-Puig A, Casillas J, Bernadó-Mansilla E (2008a) A comparative study of several genetic-based classifiers in supervised learning. In: Learning classifier systems in data mining. Studies in computational intelligence, chap 10, vol 125. Springer, Berlin, pp 205–230Google Scholar
  64. Orriols-Puig A, Sastry K, Goldberg D, Bernadó-Mansilla E (2008b) Substructural surrogates for learning decomposable classification problems. In: Bacardit J, Bernadó-Mansilla E, Butz M, Kovacs T, Llorà à X, Takadama K (eds) Learning classifier systems. Lecture notes in computer science, vol 4998, Springer, Berlin, pp 235–254. doi:10.1007/978-3-540-88138-4_14
  65. Platt JC (1999) Fast training of support vector machines using sequential minimal optimization, MIT Press, Cambridge, pp 185–208Google Scholar
  66. Quinlan JR (1993) C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San FranciscoGoogle Scholar
  67. Rissanen J (1978) Modeling by shortest data description. Automatica 14:465–471MATHCrossRefGoogle Scholar
  68. Sarafis IA (2005) Data mining clustering of high dimensional databases with evolutionary algorithms. PhD thesis, Deptartment of Computer Science, School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, Scotland, UKGoogle Scholar
  69. Sastry K (2005) Principled efficiency enhancement techniques. In: Genetic and Evolutionary Computation Conference-GECCO 2005-Tutorial. Available at url:http://www.illigal.uiuc.edu/web/kumara/2005/11/24/principled-efficiency-enhancement-techniques/
  70. Smith SF (1980) A learning system based on genetic adaptive algorithms. PhD thesis, University of Pittsburgh, url:http://portal.acm.org/citation.cfm?id=909835
  71. Smith SF (1983) Flexible learning of problem solving heuristics through adaptive search. In: Proceedings of the eighth international joint conference on artificial intelligence, vol 1. Morgan Kaufmann Publishers Inc., Karlsruhe, pp 422–425Google Scholar
  72. Smith R, Jiang M, Bacardit J, Stout M, Krasnogor N, Hirst J (2010) A learning classifier system with mutual-information-based fitness. Evolut Intell 3(1):31–50CrossRefGoogle Scholar
  73. Stout M, Bacardit J, Hirst JD, Krasnogor N (2008) Prediction of recursive convex hull class assignments for protein residues. Bioinformatics 24(7):916–923CrossRefGoogle Scholar
  74. Stout M, Bacardit J, Hirst JD, Smith RE, Krasnogor N (2009) Prediction of topological contacts in proteins using learning classifier systems. Soft Comput 13:245–258CrossRefGoogle Scholar
  75. Tabacman M, Bacardit J, Loiseau I, Krasnogor N (2008) Learning classifier systems in optimisation problems: a case study on fractal travelling salesman problems. In: Proceedings of the international workshop on learning classifier systems, Springer, Lecture Notes in Computer Science, vol (to appear)Google Scholar
  76. Urbanowicz R, Moore J (2010) The application of pittsburgh-style learning classifier systems to address genetic heterogeneity and epistasis in association studies. In: Schaefer R, Cotta C, Kolodziej J, Rudolph G (eds) Parallel problem solving from nature-PPSN XI. Lecture notes in computer science, chap 41, vol 6238. Springer, Berlin, pp 404–413Google Scholar
  77. Urbanowicz RJ, Moore JH (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1–25CrossRefGoogle Scholar
  78. Venturini G (1993) SIA: a supervised inductive algorithm with genetic search for learning attributes based concepts. In: Brazdil PB (ed) Machine Learning: ECML-93—Proceedings of the European conference on machine learning. Springer, Berlin, pp 280–296Google Scholar
  79. Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1(6):80–83CrossRefGoogle Scholar
  80. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRefGoogle Scholar
  81. Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Learning classifier systems. From foundations to applications. LNAI-1813, Springer, Berlin, pp 209–219Google Scholar
  82. Wilson SW (2001) Mining oblique data with XCS. In: Luca Lanzi P, Stolzmann W, Wilson S (eds) Advances in learning classifier systems. Lecture notes in computer science, vol 1996, Springer, Berlin, pp 283–290Google Scholar
  83. Wilson SW (2002) Classifiers that approximate functions. Natural Comput 1(2–3):211–234MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • María A. Franco
    • 1
  • Natalio Krasnogor
    • 1
  • Jaume Bacardit
    • 1
    • 2
  1. 1.ICOS Research Group, School of Computer ScienceUniversity of NottinghamNottinghamUK
  2. 2.Multi-disciplinary Centre for Integrative Biology (MyCIB), School of BiosciencesUniversity of NottinghamSutton BoningtonUK

Personalised recommendations