Evolutionary Intelligence

, Volume 6, Issue 2, pp 127–134 | Cite as

A multi-core parallelization strategy for statistical significance testing in learning classifier systems

  • James Rudd
  • Jason H. Moore
  • Ryan J. Urbanowicz
Research Paper


Permutation-based statistics for evaluating the significance of class prediction, predictive attributes, and patterns of association have only appeared within the learning classifier system (LCS) literature since 2012. While still not widely utilized by the LCS research community, formal evaluations of statistical confidence are imperative to large and complex real world applications such as genetic epidemiology where it is standard practice to quantify the likelihood that a seemingly meaningful statistic could have been obtained purely by chance. Learning classifier system algorithms are relatively computationally expensive on their own. The compounding requirements for generating permutation-based statistics may be a limiting factor for some researchers interested in applying LCS algorithms to real world problems. Technology has made LCS parallelization strategies more accessible and thus more popular in recent years. In the present study we examine the benefits of externally parallelizing a series of independent LCS runs such that permutation testing with cross validation becomes more feasible to complete on a single multi-core workstation. We test our python implementation of this strategy in the context of a simulated complex genetic epidemiological data mining problem. Our evaluations indicate that as long as the number of concurrent processes does not exceed the number of CPU cores, the speedup achieved is approximately linear.


LCS Significance testing Parallelization Scalability Multi-core processors 



This work was supported by NIH grants LM011360, LM009012 and LM010098.


  1. 1.
    Genetics based machine learning central.
  2. 2.
  3. 3.
  4. 4.
    Bacardit J, Llorà X (2013) Large-scale data mining using genetics-based machine learning. Wiley Interdiscip Rev Data Min Knowl Discov 3(1):37–61CrossRefGoogle Scholar
  5. 5.
    Bernadó-Mansilla E, Garrell-Guiu JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238CrossRefGoogle Scholar
  6. 6.
    Binet S, Calafiura P, Snyder S, Wiedenmann W, Winklmeier F (2010) Harnessing multicores: strategies and implementations in atlas. J Phys Conf Ser 219:042002. IOP PublishingGoogle Scholar
  7. 7.
    Foley SS, Elwasif WR, Bernholdt DE (2011) The integrated plasma simulator: a flexible python framework for coupled multiphysics simulation. PyHPC 2011: Python for High Performance and Scientific ComputingGoogle Scholar
  8. 8.
    Friborg RM, Bjørndalen JM, Vinter B (2009) Three unique implementations of processes for pycsp. Commun Process Archit 2009:277–292Google Scholar
  9. 9.
    Lanzi PL, Loiacono D (2010) Speeding up matching in learning classifier systems using cuda. Learn Classif Syst 1–20. SpringerGoogle Scholar
  10. 10.
    Loiacono D (2011) Fast prediction computation in learning classifier systems using cuda. In: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation, pp 169–170. ACMGoogle Scholar
  11. 11.
    Moore JH, Asselbergs FW, Williams SM (2010) Bioinformatics challenges for genome-wide association studies. Bioinformatics 26(4):445–455CrossRefGoogle Scholar
  12. 12.
    Urbanowicz R, Granizo-Mackenzie A, Moore J (2012) Instance-linked attribute tracking and feedback for michigan-style supervised learning classifier systems. In: Proceedings of the fourteenth international conference on genetic and evolutionary computation conference, pp 927–934. ACMGoogle Scholar
  13. 13.
    Urbanowicz RJ, Andrew AS, Karagas MR, Moore JH (2013) Role of genetic heterogeneity and epistasis in bladder cancer susceptibility and outcome: a learning classifier system approach. J AMIAGoogle Scholar
  14. 14.
    Urbanowicz RJ, Granizo-Mackenzie A, Moore JH (2012) An analysis pipeline with statistical and visualization-guided knowledge discovery for michigan-style learning classifier systems. Comput Intell Mag IEEE 7(4):35–45CrossRefGoogle Scholar
  15. 15.
    Urbanowicz RJ, Kiralis J, Fisher JM, Moore JH (2012) Predicting the difficulty of pure, strict, epistatic models: metrics for simulated model selection. BioData Min 5(1):1–13CrossRefGoogle Scholar
  16. 16.
    Urbanowicz RJ, Kiralis J, Sinnott-Armstrong NA, Heberling T, Fisher JM, Moore JH (2012) Gametes: a fast, direct algorithm for generating pure, strict, epistatic models with random architectures. BioData Min 5(1):16CrossRefGoogle Scholar
  17. 17.
    Urbanowicz RJ, Moore JH (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1CrossRefGoogle Scholar
  18. 18.
    Urbanowicz RJ, Moore JH (2010) The application of michigan-style learning classifiersystems to address genetic heterogeneity and epistasisin association studies. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, pp 195–202. ACMGoogle Scholar
  19. 19.
    Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • James Rudd
    • 1
  • Jason H. Moore
    • 1
  • Ryan J. Urbanowicz
    • 1
  1. 1.Dartmouth CollegeLebanonUSA

Personalised recommendations