Single-Class Classification with Mapping Convergence

Abstract

Single-Class Classification (SCC) seeks to distinguish one class of data from universal set of multiple classes. We call the target class positive and the complement set of samples negative. In SCC problems, it is assumed that a reasonable sample of the negative data is not available. SCC problems are prevalent in the real world where positive and unlabeled data are widely available but negative data are hard or expensive to acquire. We present an SCC algorithm called Mapping Convergence (MC) that computes an accurate boundary of the target class from positive and unlabeled data (without labeled negative data). The basic idea of MC is to exploit the natural “gap” between positive and negative data by incrementally labeling negative data from the unlabeled data using the margin maximization property of SVM. We also present Support Vector Mapping Convergence (SVMC) which optimizes the MC algorithm for fast training. Our analyses show that MC and SVMC without labeled negative data significantly outperform other SCC methods. They generate as accurate boundaries as standard SVM with fully labeled data when the positive data is not very under-sampled and there exist gaps between positive and negative classes in the feature space. Our results also show that SVMC trains much faster than MC with very close accuracy.

References

  1. Bileschi, S. M., & Heisele, B. (2003). Advances in component-based face detection. In IEEE Int. Workshop on Analysis and Modeling of Faces and Gestures (pp. 149–156).

  2. Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, 121–167.

    Article  Google Scholar 

  3. Chang, C.-C., & Lin, C.-J. (2001). Training nu-support vector classifiers: Theory and algorithms. Neural Computation, 13, 2119–2147.

    Article  Google Scholar 

  4. Christianini, N., & Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press.

  5. DeComite, F., Denis, F., & Gilleron, R. (1999). Positive and unlabeled examples help learning. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'99) (pp. 219–230).

  6. Denis, F. (1998). PAC Learning from positive statistical queries. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'99) (pp. 112–126).

  7. Frosini, A., Gori, M., & Priami, P. (1996). A neural network-based model for paper currency recognition and verification. IEEE Transactions on Neural Networks, 7:6, 1482–1490.

    Article  Google Scholar 

  8. Gori, M., Lastrucci, L., & Soda, G. (1995). Autoassociator-based models for speaker verification. Pattern Recognition Letters, 17, 241–250.

    Google Scholar 

  9. Joachims, T. (2001). A statistical learning model of text classification with support vector machines. In Proc. ACM SIGIR Int. Conf. Information Retrieval (SIGIR'01) (pp. 128–136).

  10. Letouzey, F., Denis, F., & Gilleron, R. (2000). Learning from positive and unlabeled examples. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'00) (pp. 11–30).

  11. Liu, B., Lee, W. S., Yu, P. S., & Li, X. (2002). Partially supervised classification of text documents. In Proc. Int. Conf. Machine Learning (ICML'02) (pp. 387–394).

  12. Manevitz, L. M., & Yousef, M. (2001). One-class SVMs for document classification. Journal of Machine Learning Research, 2, 139–154.

    Google Scholar 

  13. Scholkopf, B., Platt, J., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, B:7 1443–1471.

    Google Scholar 

  14. Scholkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12, 1083–1121.

    Google Scholar 

  15. Tax, D. M. J., & Duin, R. P. W. (2001). Uniform object generation for optimizing one-class classifiers. Journal of Machine Learning Research, 2, 155–173.

    Google Scholar 

  16. Tong, S. and Koller, D. (2000). Support vector machine active learning with applications to text classification. In Proc. Int. Conf. Machine Learning (ICML'00) (pp. 999–1006).

  17. Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27, 1134–1142.

    Article  MATH  Google Scholar 

  18. Vapnik, V. N. (1998). Statistical Learning Theory. John Wiley and Sons.

  19. Yu, H. (2003). SVMC: Single-class classification with support vector machines. In Proc. Int. Joint Conf. on Articial Intelligence (IJCAI'03) (pp. 567–572).

  20. Yu, H., Han, J., & Chang, K. C. (2002). PEBL: Positive-example based learning for web page classification using SVM. In Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD'02) (pp. 239–248).

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hwanjo Yu.

Additional information

This paper is based on the paper: H. Yu “SVMC: Single-Class Classification With Support Vector Machines”, Proc. 2003 International Joint Conferences on Artificial Intelligence (IJCAI'03). However, this submission is substantially extended and revised in technical contents and contains new experimental results and discussions in comparison with our conference publication.

Editor: Philip M. Long

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yu, H. Single-Class Classification with Mapping Convergence. Mach Learn 61, 49–69 (2005). https://doi.org/10.1007/s10994-005-1122-7

Download citation

Keywords

  • Single-Class Classification
  • Support Vector Machines