nPool: Massively Distributed Simultaneous Evolution and Cross-Validation in EC-Star
We introduce a cross-validation algorithm called nPool that can be applied in a distributed fashion. Unlike classic k-fold cross-validation, the data segments are mutually exclusive, and training takes place only on one segment. This system is well suited to run in concert with the EC-Star distributed Evolutionary system, cross-validating solution candidates during a run. The system is tested with different numbers of validation segments using a real-world problem of classifying ICU blood-pressure time series.
KeywordsEvolutionary computation Distributed processing Machine learning Cross-validation
The authors wish to thank Sentient Technologies for sponsoring this research and providing the processing capacity required for the experiments presented in this paper.
- Hornby, GS (2006) ALPS: the age-layered population structure for reducing the problem of premature convergence. In: Proceedings of the 8th annual conference on Genetic and evolutionary computation, ACM, New York, pp 815–822Google Scholar
- Refaeilzadeh P, Tang L, Liu H (2009) Cross-validation. In: Encyclopedia of database systems. Springer, Berlin, pp 532–538Google Scholar
- Rivest RL (1987) Learning decision lists. Mach Learn 2(3):229–246Google Scholar