Kernel Matrix Learning for One-Class Classification
Kernel-based one-class classification is a special type of classification problem, and is widely used as the outlier detection and novelty detection technique. One of the most commonly used method is the support vector dada description (SVDD). However, the performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. In this paper, we focus on the problem of choosing the optimal kernel from a kernel convex hull for the given one-class classification task, and propose a new approach. Kernel methods work by nonlinearly mapping the data into an embedding feature space, and then searching the relations among this space, however this mapping is implicitly performed by the kernel function. How to choose a suitable kernel is a difficult problem. In our method, we first transform the data points linearly so that we obtain a new set whose variances equal unity. Then we choose the minimum embedding ball as the criterion to learn the optimal kernel matrix over the kernel convex hull. It leads to the convex quadratically constrained quadratic programming (QCQP). Experiments results on a collection of benchmark data sets demonstrated the effectiveness of the proposed method.
KeywordsOne-class kernel matrix learning Kernel learning One-class classification Kernel selection Support vector data description
Unable to display preview. Download preview PDF.
- 1.Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple Lernel Learning, Conic Duality, and the SMO Algorithm. In: Proceedings of the International Conference on Machine Learning (2004)Google Scholar
- 5.Cristianini, N., Shawe-Taylor, J., Elisseeff, A., Kandola, J.: On Kernel-Target Alignment. Advances in Neural Information Processing Systems 14 (2001)Google Scholar
- 7.Hoi, S.C.H., Jin, R., Lyu, M.R.: Learning Nonparametric Kernel Matrices from Pairwise Constraints. In: Proceedings of the 24th International Conference on Machine Learning, pp. 361–368 (2007)Google Scholar
- 8.Lanckriet, G.R.G., Cristianini, N., Bartlett, P., El, G.L., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. The Journal of Machine Learning Research 5, 27–72 (2004)Google Scholar
- 10.Nguyen, C.H., Ho, T.B.: Kernel Matrix Evaluation. In: International Joint Conference on Artificial Intelligence, vol. 20, pp. 987–992 (2007)Google Scholar
- 12.Platt, J.C.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Advances in kernel methods: support vector learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
- 13.Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
- 14.Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
- 15.Sonnenburg, S., Rtsch, G., Schfer, C., Schlköpf, B.: Large Scale Multiple Kernel Learning. Journal of Machine Learning Research 7, 1531–1565 (2006)Google Scholar
- 17.Tax, D.M.J., Duin, R.P.W., Arzhaeva, Y.: Linear Model Combining by Optimizing the Area under the ROC Curve. In: Proceedings of the 18th International Conference on Pattern Recognition (2006)Google Scholar
- 21.Ye, J., Ji, S., Chen, J.: Learning the Kernel Matrix in Discriminant Analysis via Quadratically Constrained Quadratic Programming. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 854–863 (2007)Google Scholar