Single-Class Classification (SCC) seeks to distinguish one class of data from universal set of multiple classes. We call the target class positive and the complement set of samples negative. In SCC problems, it is assumed that a reasonable sample of the negative data is not available. SCC problems are prevalent in the real world where positive and unlabeled data are widely available but negative data are hard or expensive to acquire. We present an SCC algorithm called Mapping Convergence (MC) that computes an accurate boundary of the target class from positive and unlabeled data (without labeled negative data). The basic idea of MC is to exploit the natural “gap” between positive and negative data by incrementally labeling negative data from the unlabeled data using the margin maximization property of SVM. We also present Support Vector Mapping Convergence (SVMC) which optimizes the MC algorithm for fast training. Our analyses show that MC and SVMC without labeled negative data significantly outperform other SCC methods. They generate as accurate boundaries as standard SVM with fully labeled data when the positive data is not very under-sampled and there exist gaps between positive and negative classes in the feature space. Our results also show that SVMC trains much faster than MC with very close accuracy.
Bileschi, S. M., & Heisele, B. (2003). Advances in component-based face detection. In IEEE Int. Workshop on Analysis and Modeling of Faces and Gestures (pp. 149–156).
Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2, 121–167.
Chang, C.-C., & Lin, C.-J. (2001). Training nu-support vector classifiers: Theory and algorithms. Neural Computation, 13, 2119–2147.
Christianini, N., & Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press.
DeComite, F., Denis, F., & Gilleron, R. (1999). Positive and unlabeled examples help learning. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'99) (pp. 219–230).
Denis, F. (1998). PAC Learning from positive statistical queries. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'99) (pp. 112–126).
Frosini, A., Gori, M., & Priami, P. (1996). A neural network-based model for paper currency recognition and verification. IEEE Transactions on Neural Networks, 7:6, 1482–1490.
Gori, M., Lastrucci, L., & Soda, G. (1995). Autoassociator-based models for speaker verification. Pattern Recognition Letters, 17, 241–250.
Joachims, T. (2001). A statistical learning model of text classification with support vector machines. In Proc. ACM SIGIR Int. Conf. Information Retrieval (SIGIR'01) (pp. 128–136).
Letouzey, F., Denis, F., & Gilleron, R. (2000). Learning from positive and unlabeled examples. In Proc. Int. Conf. Algorithmic Learning Theory (ALT'00) (pp. 11–30).
Liu, B., Lee, W. S., Yu, P. S., & Li, X. (2002). Partially supervised classification of text documents. In Proc. Int. Conf. Machine Learning (ICML'02) (pp. 387–394).
Manevitz, L. M., & Yousef, M. (2001). One-class SVMs for document classification. Journal of Machine Learning Research, 2, 139–154.
Scholkopf, B., Platt, J., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, B:7 1443–1471.
Scholkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New support vector algorithms. Neural Computation, 12, 1083–1121.
Tax, D. M. J., & Duin, R. P. W. (2001). Uniform object generation for optimizing one-class classifiers. Journal of Machine Learning Research, 2, 155–173.
Tong, S. and Koller, D. (2000). Support vector machine active learning with applications to text classification. In Proc. Int. Conf. Machine Learning (ICML'00) (pp. 999–1006).
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27, 1134–1142.
Vapnik, V. N. (1998). Statistical Learning Theory. John Wiley and Sons.
Yu, H. (2003). SVMC: Single-class classification with support vector machines. In Proc. Int. Joint Conf. on Articial Intelligence (IJCAI'03) (pp. 567–572).
Yu, H., Han, J., & Chang, K. C. (2002). PEBL: Positive-example based learning for web page classification using SVM. In Proc. ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD'02) (pp. 239–248).
This paper is based on the paper: H. Yu “SVMC: Single-Class Classification With Support Vector Machines”, Proc. 2003 International Joint Conferences on Artificial Intelligence (IJCAI'03). However, this submission is substantially extended and revised in technical contents and contains new experimental results and discussions in comparison with our conference publication.
Editor: Philip M. Long
About this article
Cite this article
Yu, H. Single-Class Classification with Mapping Convergence. Mach Learn 61, 49–69 (2005). https://doi.org/10.1007/s10994-005-1122-7
- Single-Class Classification
- Support Vector Machines