One-Class Support Vector Machines and Density Estimation: The Precise Relation

  • Alberto Muñoz
  • Javier M. Moguerza
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3287)


One-Class Support Vector Machines (SVM) afford the problem of estimating high density regions from univariate or multivariate data samples. To be more precise, sets whose probability is specified in advance are estimated. In this paper the exact relation between One-Class SVM and density estimation is demonstrated. This relation provides theoretical background for the behaviour of One-Class SVM when the Gaussian kernel is used, the only case for which successful results are shown in the literature.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alpaydin, E., Kaynak, C.: Cascading Classifiers. Kybernetika 34(4), 369–374 (1998)Google Scholar
  2. 2.
    Aroszajn, N.: Theory of Reproducing Kernels. Transactions of the American Mathematical Society 68(3), 337–404 (1950)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms, 2nd edn. Wiley, New York (1993)MATHGoogle Scholar
  4. 4.
    Ben-David, S., Lindenbaum, M.: Learning distributions by their density levels: a paradigm for learning without a teacher. Journal of Computer and System Sciences 55, 171–182 (1997)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Ben-Hur, A., Horn, D., Siegelmann, H., Vapnik, V.: Support Vector Clustering. Journal of Machine Learning Research 2, 125–137 (2001)CrossRefGoogle Scholar
  6. 6.
    Cuevas, A., Fraiman, R.: A plug-in approach to support estimation. The Annals of Statistics 25(6), 2300–2312 (1997)MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Devroye, L.: Recursive estimation of the mode of a multivariate density. The Canadian Journal of Statistics 7(2), 159–167 (1979)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Devroye, L., Wise, G.: Detection of abnormal behavior via nonparametric estimation of the support. SIAM J. Appl. Math. 38, 480–488 (1980)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Kimeldorf, G.S., Wahba, G.: A Correspondence between Bayesian Estimation on Stochastic Processes and Smoothing by Splines. Annals of Mathematical Statistics 2, 495–502 (1971)MathSciNetGoogle Scholar
  10. 10.
    Moguerza, J.M., Muñoz, A., Martin-Merino, M.: Detecting the Number of Clusters Using a Support Vector Machine Approach. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 763–768. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Muñoz, A., Muruzabal, J.: Self-Organizing Maps for Outlier Detection. Neurocomputing 18, 33–60 (1998)CrossRefGoogle Scholar
  12. 12.
    Rätsch, G., Mika, S., Schölkopf, B., Müller, K.R.: Constructing Boosting Algorithms from SVMs: an Application to One-Class Classification. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(9), 1184–1199 (2002)CrossRefGoogle Scholar
  13. 13.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the Support of a High Dimensional Distribution. Neural Computation 13(7), 1443–1471 (2001)MATHCrossRefGoogle Scholar
  14. 14.
    Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, Boca Raton (1990)Google Scholar
  15. 15.
    Tax, D.M.J., Duin, R.P.W.: Support Vector Domain Description. Pattern Recognition Letters 20, 1991–1999 (1999)CrossRefGoogle Scholar
  16. 16.
    Tikhonov, A.N., Arsenin, V.Y.: Solutions of ill-posed problems. John Wiley & Sons, New York (1977)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Alberto Muñoz
    • 1
  • Javier M. Moguerza
    • 2
  1. 1.University Carlos IIIGetafeSpain
  2. 2.University Rey Juan CarlosMóstolesSpain

Personalised recommendations