Ensembles of One Class Support Vector Machines

  • Albert D. Shieh
  • David F. Kamm
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5519)


The one class support vector machine (OCSVM) is a widely used approach to one class classification, the problem of distinguising one class of data from the rest of the feature space. However, even with optimal parameter selection, the OCSVM can be sensitive to overfitting in the presence of noise. Bagging is an ensemble method that can reduce the influence of noise and prevent overfitting. In this paper, we propose a bagging OCSVM using kernel density estimation to decrease the weight given to noise. We demonstrate the improved performance of the bagging OCSVM on both simulated and real world data sets.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36, 105–139 (1999)CrossRefGoogle Scholar
  2. 2.
    Bicego, M., Figueiredo, M.A.T.: Soft clustering using weighted one-class support vector machines. Pattern Recognition 42, 27–32 (2009)CrossRefzbMATHGoogle Scholar
  3. 3.
    Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHGoogle Scholar
  4. 4.
    Di Marzio, M., Taylor, C.C.: Boosting kernel density estimates: a bias reduction technique? Biometrika 91, 226–233 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Hempstalk, K., Frank, E., Witten, I.H.: One-class classification by combining density and class probability estimation. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part I. LNCS (LNAI), vol. 5211, pp. 505–519. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Hoffmann, H.: Kernel PCA for novelty detection. Pattern Recognition 40, 863–874 (2007)CrossRefzbMATHGoogle Scholar
  7. 7.
    Parzen, E.: On estimation of a probability density function and mode. Annals of Mathematical Statistics 33, 1065–1076 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Perdisci, R., Gu, G., Lee, W.: Using an ensemble of one-class SVM classifiers to harden payload-based anomaly detection systems. In: 6th IEEE International Conference on Data Mining, pp. 488–498. IEEE Press, New York (2006)Google Scholar
  9. 9.
    Li, C., Zhang, Y.: Bagging one-class decision trees. In: 5th International Conference on Fuzzy Systems and Knowledge Discovery, pp. 420–423. IEEE Press, New York (2008)Google Scholar
  10. 10.
    Roth, V.: Kernel fisher discriminants for outlier detection. Neural Computation 18, 942–960 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Scholköpf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Computation 13, 1443–1471 (2001)CrossRefzbMATHGoogle Scholar
  12. 12.
    Tax, D.M.J., Duin, R.P.W.: Support vector domain description. Pattern Recognition Letters 20, 1191–1999 (1999)CrossRefGoogle Scholar
  13. 13.
    Tax, D.M.J.: One-class classification. Ph.D thesis, Delft University of Technology (2001)Google Scholar
  14. 14.
    Tax, D.M.J., Duin, R.P.W.: Combining one-class classifiers. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 299–308. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  15. 15.
    Tax, D.M.J., Juszczak, P.: Kernel whitening for one-class classification. In: Lee, S.-W., Verri, A. (eds.) SVM 2002. LNCS, vol. 2388, pp. 40–52. Springer, Heidelberg (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Albert D. Shieh
    • 1
  • David F. Kamm
    • 2
  1. 1.Department of StatisticsHarvard UniversityCambridgeUSA
  2. 2.Department of Computer ScienceStanford UniversityStanfordUSA

Personalised recommendations