Pruning Ensembles of One-Class Classifiers with X-means Clustering

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9011)


In this paper, we present a novel approach for pruning ensembles of one-class classifiers. One-class classification is among the most challenging topics in the contemporary machine learning. Creating multiple classifier systems for this task is one of the most effective ways of improving the quality and robustness in case of lack of counterexamples. However, very often we are faced with the problem of redundant or weak classifiers in the pool, as one-class ensembles tend to overproduce the base learners. To tackle this problem a dedicated pruning scheme must be employed, which will allow to discard classifiers that do not contribute to the formed ensemble. We propose to approach this problem as a clustering task. We discover groups of classifiers according to their support function values for the target class. For each group, we select the most representative classifier and discard the remaining ones. We apply an efficient x-means clustering algorithm, that automatically establishes the optimal number of clusters with the use of the Bayesian Information Criterion. Experimental results carried out on a set of benchmarks prove, that our proposed method is able to provide an efficient pruning mechanism for one-class problems.


Machine learning One-class classification Classifier ensemble Ensemble pruning Clustering X-means 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alpaydin, E.: Combined 5 x 2 cv f test for comparing supervised classification learning algorithms. Neural Computation 11(8), 1885–1892 (1999)CrossRefGoogle Scholar
  2. 2.
    Cheplygina, V., Tax, D.M.J.: Pruned random subspace method for one-class classifiers. In: Sansone, C., Kittler, J., Roli, F. (eds.) MCS 2011. LNCS, vol. 6713, pp. 96–105. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  3. 3.
    Cyganek, B.: One-class support vector ensembles for image segmentation and classification. Journal of Mathematical Imaging and Vision 42(2–3), 103–117 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)zbMATHMathSciNetGoogle Scholar
  5. 5.
    Galar, M., Fernández, A., Barrenechea Tartas, E., Bustince Sola, H., Herrera, F.: Dynamic classifier selection for one-vs-one strategy: Avoiding non-competent classifiers. Pattern Recognition 46(12), 3412–3424 (2013)CrossRefGoogle Scholar
  6. 6.
    García, S., Fernández, A., Luengo, J., Herrera, F.: Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf. Sci. 180(10), 2044–2064 (2010)CrossRefGoogle Scholar
  7. 7.
    Kang, P., Kim, D., Cho, S.: Evaluating the reliability level of virtual metrology results for flexible process control: a novelty detection-based approach. Pattern Analysis and Applications 17(4), 863–881 (2014)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Krawczyk, B., Woźniak, M.: Combining diverse one-class classifiers. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012, Part II. LNCS, vol. 7209, pp. 590–601. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  9. 9.
    Krawczyk, B., Woźniak, M.: Diversity measures for one-class classifier ensembles. Neurocomputing 126, 36–44 (2014)CrossRefGoogle Scholar
  10. 10.
    Pelleg, D., Moore, A.W.: X-means: extending k-means with efficient estimation of the number of clusters. In: Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000), June 29 - July 2, 2000, pp. 727–734. Stanford University, Stanford (2000)Google Scholar
  11. 11.
    Tax, D.M.J., Duin, R.P.W.: Support vector data description. Machine Learning 54(1), 45–66 (2004)CrossRefzbMATHGoogle Scholar
  12. 12.
    Tax, D.M.J., Muller, K.: A consistency-based model selection for one-class classification. In: Proceedings - International Conference on Pattern Recognition, vol. 3, pp. 363–366 (2004). Cited By (since 1996):12Google Scholar
  13. 13.
    Woźniak, M., Grana, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Information Fusion 16(1), 3–17 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWrocław University of TechnologyWrocławPoland

Personalised recommendations