Kernel Matrix Learning for One-Class Classification

  • Chengqun Wang
  • Jiangang Lu
  • Chonghai Hu
  • Youxian Sun
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5263)


Kernel-based one-class classification is a special type of classification problem, and is widely used as the outlier detection and novelty detection technique. One of the most commonly used method is the support vector dada description (SVDD). However, the performance is mostly affected by which kernel is used. A promising way is to learn the kernel from the data automatically. In this paper, we focus on the problem of choosing the optimal kernel from a kernel convex hull for the given one-class classification task, and propose a new approach. Kernel methods work by nonlinearly mapping the data into an embedding feature space, and then searching the relations among this space, however this mapping is implicitly performed by the kernel function. How to choose a suitable kernel is a difficult problem. In our method, we first transform the data points linearly so that we obtain a new set whose variances equal unity. Then we choose the minimum embedding ball as the criterion to learn the optimal kernel matrix over the kernel convex hull. It leads to the convex quadratically constrained quadratic programming (QCQP). Experiments results on a collection of benchmark data sets demonstrated the effectiveness of the proposed method.


One-class kernel matrix learning Kernel learning One-class classification Kernel selection Support vector data description 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple Lernel Learning, Conic Duality, and the SMO Algorithm. In: Proceedings of the International Conference on Machine Learning (2004)Google Scholar
  2. 2.
    Boyd, S.P., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)zbMATHGoogle Scholar
  3. 3.
    Cervantes, J., Li, X., Yu, W., Li, K.: Support Vector Machine Classification for Large Data Sets via Minimum Enclosing Ball Clustering. Neurocomputing 71, 611–619 (2008)CrossRefGoogle Scholar
  4. 4.
    Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing Multiple Parameters for Support Vector Machines. Machine Learning 46, 131–159 (2002)zbMATHCrossRefGoogle Scholar
  5. 5.
    Cristianini, N., Shawe-Taylor, J., Elisseeff, A., Kandola, J.: On Kernel-Target Alignment. Advances in Neural Information Processing Systems 14 (2001)Google Scholar
  6. 6.
    Duan, K., Keerthi, S.S., Poo, A.N.: Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters. Neurocomputing 51, 41–59 (2003)CrossRefGoogle Scholar
  7. 7.
    Hoi, S.C.H., Jin, R., Lyu, M.R.: Learning Nonparametric Kernel Matrices from Pairwise Constraints. In: Proceedings of the 24th International Conference on Machine Learning, pp. 361–368 (2007)Google Scholar
  8. 8.
    Lanckriet, G.R.G., Cristianini, N., Bartlett, P., El, G.L., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. The Journal of Machine Learning Research 5, 27–72 (2004)Google Scholar
  9. 9.
    Meyer, D., Leisch, F., Hornik, K.: The Support Vector Machine under Test. Neurocomputing 55, 169–186 (2003)CrossRefGoogle Scholar
  10. 10.
    Nguyen, C.H., Ho, T.B.: Kernel Matrix Evaluation. In: International Joint Conference on Artificial Intelligence, vol. 20, pp. 987–992 (2007)Google Scholar
  11. 11.
    Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the Kernel with Hyperkernels. The Journal of Machine Learning Research 6, 1043–1071 (2005)MathSciNetGoogle Scholar
  12. 12.
    Platt, J.C.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Advances in kernel methods: support vector learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  13. 13.
    Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)Google Scholar
  14. 14.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  15. 15.
    Sonnenburg, S., Rtsch, G., Schfer, C., Schlköpf, B.: Large Scale Multiple Kernel Learning. Journal of Machine Learning Research 7, 1531–1565 (2006)Google Scholar
  16. 16.
    Tax, D.M.J., Duin, R.P.W.: Support Vector Data Description. Machine Learning 54, 45–66 (2004)zbMATHCrossRefGoogle Scholar
  17. 17.
    Tax, D.M.J., Duin, R.P.W., Arzhaeva, Y.: Linear Model Combining by Optimizing the Area under the ROC Curve. In: Proceedings of the 18th International Conference on Pattern Recognition (2006)Google Scholar
  18. 18.
    Tsang, I.W., Kwok, J.T., Cheung, P.M.: Core Vector Machines: Fast SVM Training on Very Large Data Sets. The Journal of Machine Learning Research 6, 363–392 (2005)MathSciNetGoogle Scholar
  19. 19.
    Tsang, I.W., Kwok, J.T., Zurada, J.M.: Generalized Core Vector Machines. IEEE Transtractions on Neural Networks 17, 1126–1140 (2006)CrossRefGoogle Scholar
  20. 20.
    Yeung, D.Y., Chang, H., Dai, G.: Learning the Kernel Matrix by Maximizing a KFD-based Class Separability Criterion. Pattern Recognition 40, 2021–2028 (2007)zbMATHCrossRefGoogle Scholar
  21. 21.
    Ye, J., Ji, S., Chen, J.: Learning the Kernel Matrix in Discriminant Analysis via Quadratically Constrained Quadratic Programming. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 854–863 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Chengqun Wang
    • 1
  • Jiangang Lu
    • 1
  • Chonghai Hu
    • 2
  • Youxian Sun
    • 1
  1. 1.State Key Lab. of Industrial Control Tech.Zhejiang UniversityChina
  2. 2.Dept. of MathematicsZhejiang UniversityChina

Personalised recommendations