Advertisement

A Reformulation of Support Vector Machines for General Confidence Functions

  • Yuhong Guo
  • Dale Schuurmans
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5828)

Abstract

We present a generalized view of support vector machines that does not rely on a Euclidean geometric interpretation nor even positive semidefinite kernels. We base our development instead on the confidence matrix—the matrix normally determined by the direct (Hadamard) product of the kernel matrix with the label outer-product matrix. It turns out that alternative forms of confidence matrices are possible, and indeed useful. By focusing on the confidence matrix instead of the underlying kernel, we can derive an intuitive principle for optimizing example weights to yield robust classifiers. Our principle initially recovers the standard quadratic SVM training criterion, which is only convex for kernel-derived confidence measures. However, given our generalized view, we are then able to derive a principled relaxation of the SVM criterion that yields a convex upper bound. This relaxation is always convex and can be solved with a linear program. Our new training procedure obtains similar generalization performance to standard SVMs on kernel-derived confidence functions, but achieves even better results with indefinite confidence functions.

Keywords

Support Vector Machine Kernel Matrix Neural Information Processing System Training Label Uniform Weighting 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Boser, B., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proceedings 5th Annual Conference on Computational Learning Theory, COLT (1992)Google Scholar
  2. 2.
    Chen, J., Ye, J.: Training SVM with indefinite kernels. In: Proceedings of the International Conference on Machine Learning, ICML (2008)Google Scholar
  3. 3.
    Chen, Y., Gupta, M., Recht, B.: Learning kernels from indefinite similarities. In: Proceedings of the International Conference on Machine Learning, ICML (2009)Google Scholar
  4. 4.
    Crisp, D.J., Burges, C.J.C.: A geometric interpretation of v-SVM classifiers. In: Advances in Neural Information Processing Systems, NIPS 2000 (2000)Google Scholar
  5. 5.
    Cristianini, N., Shawe-Taylor, J., Elisseeff, A., Kandola, J.: On kernel target-alignment. In: Advances in Neural Information Processing Systems, NIPS 2001 (2001)Google Scholar
  6. 6.
    Guo, Y., Greiner, R., Schuurmans, D.: Learning coordination classifiers. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI 2005 (2005)Google Scholar
  7. 7.
    Haasdonk, B.: Feature space interpretation of SVMs with indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(4) (2005)Google Scholar
  8. 8.
    Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L., Jordan, M.: Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research 5 (2004)Google Scholar
  9. 9.
    Lin, H.-T., Lin, C.-J.: A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods (2003)Google Scholar
  10. 10.
    Luss, R., d’Aspremont, A.: Support vector machine classification with indefinite kernels. In: Advances in Neural Information Processing Systems, NIPS 2007 (2007)Google Scholar
  11. 11.
    Ong, C.S., Mary, X., Canu, S., Smola, A.J.: Learning with non-positive kernels. In: Proceedings of the International Conference on Machine Learning, ICML (2004)Google Scholar
  12. 12.
    Schoelkopf, B., Smola, A., Williamson, R., Bartlett, P.: New support vector algorithms. Neural Computation 12(5), 1207–1245 (2000)CrossRefGoogle Scholar
  13. 13.
    Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)zbMATHGoogle Scholar
  14. 14.
    Wu, G., Chang, Y., Zhang, Z.: An analysis of transformation on non-positive semidefinite similarity matrix for kernel machines. In: Proceedings of the International Conference on Machine Learning, ICML (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Yuhong Guo
    • 1
  • Dale Schuurmans
    • 2
  1. 1.Department of Computer & Information SciencesTemple University 
  2. 2.Department of Computing ScienceUniversity of Alberta 

Personalised recommendations