Skip to main content
SpringerLink
Log in

Kernel classification using a linear programming approach

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

A support vector machine (SVM) classifier corresponds in its most basic form to a quadratic programming problem. Various linear variations of support vector classification have been investigated such as minimizing the L1-norm of the weight-vector instead of the L2-norm. In this paper we introduce a classifier where we minimize the boundary (lower envelope) of the epigraph that is generated over a set of functions, which can be interpreted as a measure of distance or slack from the origin. The resulting classifier appears to provide a generalization performance similar to SVMs while displaying a more advantageous computational complexity. The discussed formulation can also be extended to allow for cases with imbalanced data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bi, J., Bennett, K.P.: Duality, geometry, and support vector regression. In: Advances in Neural Information Processing Systems, pp. 593–600 (2002)

  2. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM (1992)

  3. Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2(2), 121–167 (1998)

    Article  Google Scholar 

  4. Chvȧtal, V.: Linear Programming. Macmillan (1983)

  5. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  6. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  7. Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2017)

  8. Gurobi Optimization Inc.: Gurobi Optimizer Reference Manual, Houston, Texas. http://www.gurobi.com (2016)

  9. Herbrich, R., Graepel, T., Campbell, C.: Robust Bayes Point Machines. In: ESANN, pp. 49–54. Citeseer (2000)

  10. Malyscheff, A.M.: Optimization issues in data analysis: an analytic center approach to kernel methods. Ph.D thesis (2001)

  11. Malyscheff, A.M., Trafalis, T.B.: An analytic center machine for regression. CEJOR 10(4), 297–334 (2002)

    MathSciNet  MATH  Google Scholar 

  12. Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London, Series A, Containing Papers of a Mathematical or Physical Character 209, 415–446 (1909)

    MATH  Google Scholar 

  13. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al.: Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12(Oct), 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  14. van Rossum, G.: Python Reference Manual. CWI, Centre for Mathematics and Computer Science, Amsterdam (1995)

    Google Scholar 

  15. Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)

    Google Scholar 

  16. Trafalis, T.B., Malyscheff, A.M.: An analytic center machine. Mach. Learn. 46(1-3), 203–223 (2002)

    Article  Google Scholar 

  17. Vapnik, V.: The Nature of Statistical Learning. Springer, New York (1995)

    Book  Google Scholar 

  18. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  19. Xanthopoulos, P., Guarracino, M.R., Pardalos, P.M.: Robust generalized eigenvalue classifier with ellipsoidal uncertainty. Ann. Oper. Res. 216(1), 327–342 (2014)

    Article  MathSciNet  Google Scholar 

  20. Xanthopoulos, P., Pardalos, P.M., Trafalis, T.B.: Robust Data Mining. Springer Science & Business Media, Berlin (2012)

    MATH  Google Scholar 

Download references

Acknowledgements

Theodore Trafalis work has been conducted at the National Research Institute University Higher School of Economics and has been supported by the RSF grant n. 14-41-00039.

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, University of Oklahoma, 110 W. Boyd, Norman, OK, 73019, USA

    Alexander M. Malyscheff

  2. School of Industrial and Systems Engineering, University of Oklahoma, 202 W. Boyd, Norman, OK, 73019, USA

    Theodore B. Trafalis

Authors
  1. Alexander M. Malyscheff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Theodore B. Trafalis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander M. Malyscheff.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malyscheff, A.M., Trafalis, T.B. Kernel classification using a linear programming approach. Ann Math Artif Intell 88, 39–51 (2020). https://doi.org/10.1007/s10472-019-09642-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-019-09642-w

Keywords

Mathematics Subject Classification (2010)

Search

Navigation