One-Class Support Vector Machines and Density Estimation: The Precise Relation

  • Alberto Muñoz
  • Javier M. Moguerza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3287)


One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions from univariate or multivariate data samples. To be more precise, sets whose probability is specified in advance are estimated. In this paper the exact relation between One-Class SVM and density estimation is demonstrated. This relation provides theoretical background for the behaviour of One-Class SVM when the Gaussian kernel is used, the only case for which successful results are shown in the literature.


Support Vector Machine Density Estimation Class Support Vector Machine Kernel Density Estimation Kernel Density Estimator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alpaydin, E., Kaynak, C.: Cascading Classifiers. Kybernetika 34(4), 369–374 (1998)Google Scholar
  2. 2.
    Aroszajn, N.: Theory of Reproducing Kernels. Transactions of the American Mathematical Society 68(3), 337–404 (1950)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 2nd edn. Wiley, New York (1993)zbMATHGoogle Scholar
  4. 4.
    Ben-David, S., Lindenbaum, M.: Learning distributions by their density levels: a paradigm for learning without a teacher. Journal of Computer and System Sciences 55, 171–182 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Ben-Hur, A., Horn, D., Siegelmann, H., Vapnik, V.: Support Vector Clustering. Journal of Machine Learning Research 2, 125–137 (2001)CrossRefGoogle Scholar
  6. 6.
    Cuevas, A., Fraiman, R.: A plug-in approach to support estimation. The Annals of Statistics 25(6), 2300–2312 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Devroye, L.: Recursive estimation of the mode of a multivariate density. The Canadian Journal of Statistics 7(2), 159–167 (1979)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Devroye, L., Wise, G.: Detection of abnormal behavior via nonparametric estimation of the support. SIAM J. Appl. Math. 38, 480–488 (1980)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Kimeldorf, G.S., Wahba, G.: A Correspondence between Bayesian Estimation on Stochastic Processes and Smoothing by Splines. Annals of Mathematical Statistics 2, 495–502 (1971)MathSciNetGoogle Scholar
  10. 10.
    Moguerza, J.M., Muñoz, A., Martin-Merino, M.: Detecting the Number of Clusters Using a Support Vector Machine Approach. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 763–768. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Muñoz, A., Muruzabal, J.: Self-Organizing Maps for Outlier Detection. Neurocomputing 18, 33–60 (1998)CrossRefGoogle Scholar
  12. 12.
    Rätsch, G., Mika, S., Schölkopf, B., Müller, K.R.: Constructing Boosting Algorithms from SVMs: an Application to One-Class Classification. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(9), 1184–1199 (2002)CrossRefGoogle Scholar
  13. 13.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the Support of a High Dimensional Distribution. Neural Computation 13(7), 1443–1471 (2001)zbMATHCrossRefGoogle Scholar
  14. 14.
    Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, Boca Raton (1990)Google Scholar
  15. 15.
    Tax, D.M.J., Duin, R.P.W.: Support Vector Domain Description. Pattern Recognition Letters 20, 1991–1999 (1999)CrossRefGoogle Scholar
  16. 16.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of ill-posed problems. John Wiley & Sons, New York (1977)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Alberto Muñoz
    • 1
  • Javier M. Moguerza
    • 2
  1. 1.University Carlos IIIGetafeSpain
  2. 2.University Rey Juan CarlosMóstolesSpain

Personalised recommendations