A multi-core parallelization strategy for statistical significance testing in learning classifier systems
Permutation-based statistics for evaluating the significance of class prediction, predictive attributes, and patterns of association have only appeared within the learning classifier system (LCS) literature since 2012. While still not widely utilized by the LCS research community, formal evaluations of statistical confidence are imperative to large and complex real world applications such as genetic epidemiology where it is standard practice to quantify the likelihood that a seemingly meaningful statistic could have been obtained purely by chance. Learning classifier system algorithms are relatively computationally expensive on their own. The compounding requirements for generating permutation-based statistics may be a limiting factor for some researchers interested in applying LCS algorithms to real world problems. Technology has made LCS parallelization strategies more accessible and thus more popular in recent years. In the present study we examine the benefits of externally parallelizing a series of independent LCS runs such that permutation testing with cross validation becomes more feasible to complete on a single multi-core workstation. We test our python implementation of this strategy in the context of a simulated complex genetic epidemiological data mining problem. Our evaluations indicate that as long as the number of concurrent processes does not exceed the number of CPU cores, the speedup achieved is approximately linear.
KeywordsLCS Significance testing Parallelization Scalability Multi-core processors
- 1.Genetics based machine learning central. http://gbml.org/.
- 2.Python multiprocessing module. http://docs.python.org/2/library/multiprocessing.html
- 3.Microsoft powershell. http://technet.microsoft.com/en-us/library/bb978526.aspx, 2012
- 6.Binet S, Calafiura P, Snyder S, Wiedenmann W, Winklmeier F (2010) Harnessing multicores: strategies and implementations in atlas. J Phys Conf Ser 219:042002. IOP PublishingGoogle Scholar
- 7.Foley SS, Elwasif WR, Bernholdt DE (2011) The integrated plasma simulator: a flexible python framework for coupled multiphysics simulation. PyHPC 2011: Python for High Performance and Scientific ComputingGoogle Scholar
- 8.Friborg RM, Bjørndalen JM, Vinter B (2009) Three unique implementations of processes for pycsp. Commun Process Archit 2009:277–292Google Scholar
- 9.Lanzi PL, Loiacono D (2010) Speeding up matching in learning classifier systems using cuda. Learn Classif Syst 1–20. SpringerGoogle Scholar
- 10.Loiacono D (2011) Fast prediction computation in learning classifier systems using cuda. In: Proceedings of the 13th annual conference companion on Genetic and evolutionary computation, pp 169–170. ACMGoogle Scholar
- 12.Urbanowicz R, Granizo-Mackenzie A, Moore J (2012) Instance-linked attribute tracking and feedback for michigan-style supervised learning classifier systems. In: Proceedings of the fourteenth international conference on genetic and evolutionary computation conference, pp 927–934. ACMGoogle Scholar
- 13.Urbanowicz RJ, Andrew AS, Karagas MR, Moore JH (2013) Role of genetic heterogeneity and epistasis in bladder cancer susceptibility and outcome: a learning classifier system approach. J AMIAGoogle Scholar
- 18.Urbanowicz RJ, Moore JH (2010) The application of michigan-style learning classifiersystems to address genetic heterogeneity and epistasisin association studies. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, pp 195–202. ACMGoogle Scholar