Optimization Algorithms for One-Class Classification Ensemble Pruning

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8398)


One-class classification is considered as one of the most challenging topics in the contemporary machine learning. Creating Multiple Classifier Systems for this task has proven itself as a promising research direction. Here arises a problem on how to select valuable members to the committee - so far a largely unexplored area in one-class classification. Recently, a novel scheme utilizing a multi-objective ensemble pruning was proposed. It combines selecting best individual classifiers with maintaining the diversity of the committee pool. As it relies strongly on the search algorithm applied, we investigate here the performance of different methods. Five algorithms are examined - genetic algorithm, simulated annealing, tabu search and hybrid methods, combining the mentioned approaches in the form of memetic algorithms. Using compound optimization methods leads to a significant improvement over standard search methods. Experimental results carried on a number of benchmark datasets proves that careful examination of the search algorithms for one-class ensemble pruning may greatly contribute to the quality of the committee being formed.


machine learning one-class classification classifier ensemble ensemble pruning classifier selection diversity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alpaydin, E.: Combined 5 x 2 cv f test for comparing supervised classification learning algorithms. Neural Computation 11(8), 1885–1892 (1999)CrossRefGoogle Scholar
  2. 2.
    Bi, Y.: The impact of diversity on the accuracy of evidential classifier ensembles. International Journal of Approximate Reasoning 53(4), 584–607 (2012)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Bishop, C.M.: Novelty detection and neural network validation. IEE Proceedings: Vision, Image and Signal Processing 141(4), 217–222 (1994)Google Scholar
  4. 4.
    Cheplygina, V., Tax, D.M.J.: Pruned random subspace method for one-class classifiers. In: Sansone, C., Kittler, J., Roli, F. (eds.) MCS 2011. LNCS, vol. 6713, pp. 96–105. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Czogala, E., Leski, J.: Application of entropy and energy measures of fuzziness to processing of ecg signal. Fuzzy Sets and Systems 97(1), 9–18 (1998)CrossRefGoogle Scholar
  6. 6.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9, 69–82 (2008)CrossRefGoogle Scholar
  8. 8.
    Harman, M., McMinn, P.: A theoretical and empirical study of search-based testing: Local, global, and hybrid search. IEEE Transactions on Software Engineering 36(2), 226–247 (2010)CrossRefGoogle Scholar
  9. 9.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 832–844 (1998)CrossRefGoogle Scholar
  10. 10.
    Koch, M.W., Moya, M.M., Hostetler, L.D., Fogler, R.J.: Cueing, feature discovery, and one-class learning for synthetic aperture radar automatic target recognition. Neural Networks 8(7-8), 1081–1102 (1995)CrossRefGoogle Scholar
  11. 11.
    Krawczyk, B.: Diversity in ensembles for one-class classification. In: Pechenizkiy, M., Wojciechowski, M. (eds.) New Trends in Databases & Inform. Sys. AISC, vol. 185, pp. 119–129. Springer, Heidelberg (2012)Google Scholar
  12. 12.
    Krawczyk, B., Woźniak, M.: Combining diverse one-class classifiers. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 590–601. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Krawczyk, B., Woźniak, M.: Accuracy and diversity in classifier selection for one-class classification ensembles. In: 2013 IEEE Symposium on Computational Intelligence and Ensemble Learning (CIEL), pp. 46–51 (2013)Google Scholar
  14. 14.
    Krawczyk, B., Woźniak, M.: Pruning one-class classifier ensembles by combining sphere intersection and consistency measures. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS, vol. 7894, pp. 426–436. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Krawczyk, B., Woźniak, M.: Diversity measures for one-class classifier ensembles. Neurocomputing 126, 36–44 (2014)CrossRefGoogle Scholar
  16. 16.
    Krawczyk, B., Woźniak, M., Cyganek, B.: Clustering-based ensembles for one-class classification. Information Sciences (2014)Google Scholar
  17. 17.
    SIAM. Proceedings of the Eleventh SIAM International Conference on Data Mining, SDM 2011, April 28-30, Mesa, Arizona, USA. SIAM Omnipress (2011)Google Scholar
  18. 18.
    Tax, D.M.J., Duin, R.P.W.: Support vector data description. Machine Learning 54(1), 45–66 (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Tax, D.M.J., Müller, K.: A consistency-based model selection for one-class classification. In: Proceedings - International Conference on Pattern Recognition, vol. 3, pp. 363–366 (2004)Google Scholar
  20. 20.
    Tax, D.M.J., Duin, R.P.W.: Combining one-class classifiers. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 299–308. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  21. 21.
    Tax, D.M.J., Duin, R.P.W.: Characterizing one-class datasets. In: Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, pp. 21–26 (2005)Google Scholar
  22. 22.
    Wilk, T., Woźniak, M.: Soft computing methods applied to combination of one-class classifiers. Neurocomput. 75, 185–193 (2012)CrossRefGoogle Scholar
  23. 23.
    Woźniak, M., Krawczyk, B.: Combined classifier based on feature space partitioning. Journal of Applied Mathematics and Computer Science 22(4), 855–866 (2012)MathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWrocław University of TechnologyWrocławPoland

Personalised recommendations