Optimization Algorithms for One-Class Classification Ensemble Pruning
One-class classification is considered as one of the most challenging topics in the contemporary machine learning. Creating Multiple Classifier Systems for this task has proven itself as a promising research direction. Here arises a problem on how to select valuable members to the committee - so far a largely unexplored area in one-class classification. Recently, a novel scheme utilizing a multi-objective ensemble pruning was proposed. It combines selecting best individual classifiers with maintaining the diversity of the committee pool. As it relies strongly on the search algorithm applied, we investigate here the performance of different methods. Five algorithms are examined - genetic algorithm, simulated annealing, tabu search and hybrid methods, combining the mentioned approaches in the form of memetic algorithms. Using compound optimization methods leads to a significant improvement over standard search methods. Experimental results carried on a number of benchmark datasets proves that careful examination of the search algorithms for one-class ensemble pruning may greatly contribute to the quality of the committee being formed.
Keywordsmachine learning one-class classification classifier ensemble ensemble pruning classifier selection diversity
Unable to display preview. Download preview PDF.
- 3.Bishop, C.M.: Novelty detection and neural network validation. IEE Proceedings: Vision, Image and Signal Processing 141(4), 217–222 (1994)Google Scholar
- 11.Krawczyk, B.: Diversity in ensembles for one-class classification. In: Pechenizkiy, M., Wojciechowski, M. (eds.) New Trends in Databases & Inform. Sys. AISC, vol. 185, pp. 119–129. Springer, Heidelberg (2012)Google Scholar
- 13.Krawczyk, B., Woźniak, M.: Accuracy and diversity in classifier selection for one-class classification ensembles. In: 2013 IEEE Symposium on Computational Intelligence and Ensemble Learning (CIEL), pp. 46–51 (2013)Google Scholar
- 14.Krawczyk, B., Woźniak, M.: Pruning one-class classifier ensembles by combining sphere intersection and consistency measures. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 426–436. Springer, Heidelberg (2013)CrossRefGoogle Scholar
- 16.Krawczyk, B., Woźniak, M., Cyganek, B.: Clustering-based ensembles for one-class classification. Information Sciences (2014)Google Scholar
- 17.SIAM. Proceedings of the Eleventh SIAM International Conference on Data Mining, SDM 2011, April 28-30, Mesa, Arizona, USA. SIAM Omnipress (2011)Google Scholar
- 19.Tax, D.M.J., Müller, K.: A consistency-based model selection for one-class classification. In: Proceedings - International Conference on Pattern Recognition, vol. 3, pp. 363–366 (2004)Google Scholar
- 21.Tax, D.M.J., Duin, R.P.W.: Characterizing one-class datasets. In: Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, pp. 21–26 (2005)Google Scholar