Skip to main content
SpringerLink
Log in

Support Vector Machines

  • Chapter
  • First Online:
Machine Learning for Data Science Handbook

  • 1871 Accesses

Abstract

Support vector machines (SVMs) are a set of related methods for supervised learning, applicable to both classification and regression problems. An SVM classifier creates a maximum-margin hyperplane that lies in a transformed input space and splits the example classes while maximizing the distance to the nearest cleanly split examples. The parameters of the solution hyperplane are derived from a quadratic programming optimization problem. In this chapter, we provide several formulations and discuss some key concepts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 279.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Bazaraa M. S., Sherali H. D., and Shetty C. M. Nonlinear programming: theory and algorithms. Wiley, third edition, 2006.

    Google Scholar 

  • Bertsekas D.P. Nonlinear Programming. Athena Scientific, third edition, 2016.

    Google Scholar 

  • Cai, F., Cherkassky, V.: Generalized SMO algorithm for SVM-based multitask learning. IEEE Trans. Neural Network Learning Systems 2012; 23(6):997–1003.

    Google Scholar 

  • Chang C.-C. and Lin C.-J. Training support vector classifiers: Theory and algorithms. Neural Computation 2001; 13(9):2119–2147.

    Article  MATH  Google Scholar 

  • Chang C.-C. and Lin C.-J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology. 2011; 2:(27):1–27. Software available at https://www.csie.ntu.edu.tw/~cjlin/libsvm/.

  • Chen P.-H., Lin C. -J., and Scholkopf B. A tutorial on nu-support vector machines. 2003.

    Google Scholar 

  • Chew H. G., Lim C. C., and Bogner R. E. An implementation of training dual-nu support vector machines. In Qi, Teo, and Yang, editors, Optimization and Control with Applications. Kluwer, 2003.

    Google Scholar 

  • Chung K.-M., Kao W.-C., Sun C.-L., and Lin C.-J. Decomposition methods for linear support vector machines. Neural Computation 2004; 16(8):1689–1704.

    Article  MATH  Google Scholar 

  • Cortes C. and Vapnik V. Support vector networks. Machine Learning 1995; 20:273–297.

    Article  MATH  Google Scholar 

  • Cristianini N. and Shawe-Taylor J. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, 2000.

    Book  MATH  Google Scholar 

  • Deng N., Tian Y., Zhang C. Support Vector Machines Optimization Based Theory, Algorithms, and Extensions. Chapman & Hall/CRC Data Mining and Knowledge Discovery Series, 2012.

    Google Scholar 

  • Didiot E. and Lauer F., Efficient optimization of multi-class support vector machines with MSVMpack. In Modelling, Computation and Optimization in Information Systems and Management Sciences. Springer, 23–34, 2015.

    Chapter  MATH  Google Scholar 

  • Dumais S. Using SVMs for text categorization. IEEE Intelligent Systems 1998; 13(4).

    Google Scholar 

  • Hsu C.-W. and Lin C.-J. A comparison of methods for multi-class support vector machines IEEE Transactions on Neural Networks 2002; 13(2); 415–425.

    Article  Google Scholar 

  • Hsu C.-W. Chang C.-C and Lin C.-J. A practical guide to support vector classification. 2016. Available Online: www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

  • Joachims T. Learning to Classify Text using Support Vector Machines Methods, Theory, and Algorithms. Kluwer Academic Publishers, 2002.

    Book  Google Scholar 

  • Joachims T. 2008, SVMlight, available online http://www.cs.cornell.edu/People/tj/svm_light/

  • Law M. H. and Kwok J. T. Bayesian support vector regression. Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics (AISTATS) pages 239–244, Key-West, Florida, USA, January 2000.

    Google Scholar 

  • Martin D. R., Fowlkes C. C., and Malik J. Learning to detect natural image boundaries using brightness and texture. In Advances in Neural Information Processing Systems, volume 14, 2002.

    Google Scholar 

  • Mukherjee S., Osuna E., and Girosi F. Nonlinear prediction of chaotic time series using a support vector machine. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 511–520, New-York, IEEE Press, 1997.

    Google Scholar 

  • Muller K.-R., Mika S., Ratsch G., Tsuda K., and Scholkopf B., An introduction to kernel-based learning algorithms. IEEE Neural Networks 2001; 12(2):181–201.

    Article  Google Scholar 

  • Nayak J, Naik B, and Behera H. S. A comprehensive survey on support vector machine in data mining tasks: Applications & challenges. International Journal of Database Theory and Application 2015; 8,(1):169–186.

    Article  Google Scholar 

  • Osuna E., Freund R., and Girosi F. An improved training algorithm for support vector machines. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 276–285, New-York, IEEE Press, 1997.

    Google Scholar 

  • Ratsch G., Onoda T., and Muller K.R. Soft margins for AdaBoost. Machine Learning 2001; 42(3):287–320.

    Article  MATH  Google Scholar 

  • Rifkin R. and Klautau A.. In Defense of One-vs-All Classification, Journal of Machine Learning Research 2004; 5:101–141.

    MathSciNet  MATH  Google Scholar 

  • Scholkopf B., Support Vector Learning. Oldenbourg Verlag, Munich, 1997.

    MATH  Google Scholar 

  • Scholkopf B., Burges C.J.C., and Vapnik V.N. Extracting support data for a given task. In Fayyad U.M. and Uthurusamy R., Editors, Proceedings, First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, CA, 1995.

    Google Scholar 

  • Scholkopf B., Simard P.Y., Smola A.J., and Vapnik V.N.. Prior knowledge in support vector kernels. In Jordan M., Kearns M., and Solla S., Editors, Advances in Neural Information Processing Systems 10, pages 640–646. MIT Press, Cambridge, MA, 1998.

    Google Scholar 

  • Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods - Support Vector Learning, Cambridge, MA, MIT Press, 1999.

    Google Scholar 

  • Scholkopf B. and Smola A. J. Learning with Kernels. MIT Press, Cambridge, MA, 2002.

    MATH  Google Scholar 

  • Scholkopf B., Smola A. J., Williamson R. C., and Bartlett P. L. New support vector algorithms. Neural Computation 1999; 12:1207–1245.

    Article  Google Scholar 

  • Shawe-Taylor J. and Cristianini N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.

    Book  MATH  Google Scholar 

  • Smola A. J., Bartlett P. L., Scholkopf B. and Schuurmans D. Advances in Large Margin Classifiers. MIT Press, Cambridge, MA, 2000.

    Book  MATH  Google Scholar 

  • Smola A.J. and Scholkopf B. A tutorial on support vector regression. Statistics and Computing 2004; 14(13):199–222.

    Article  MathSciNet  Google Scholar 

  • Smola A.J., Scholkopf B. and Ratsch G. Linear programs for automatic accuracy control in regression. Proceedings of International Conference on Artificial Neural Networks ICANN'99, Berlin, Springer 1999.

    Google Scholar 

  • Steinwart I. On the optimal parameter choice for nu-support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003; 23(10).

    Google Scholar 

  • Steinwart I. Sparseness of support vector machines. Journal of Machine Learning Research 2004; 4(6):1071–1105.

    MathSciNet  MATH  Google Scholar 

  • Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least Squares Support Vector Machines. World Scientific Publishing , Singapore, 2002.

    Book  MATH  Google Scholar 

  • Tavara S. Parallel Computing of Support Vector Machines: A Survey. ACM Computing Surveys, 2019;. 51(6):1–38.

    Article  Google Scholar 

  • Vapnik V. The Nature of Statistical Learning Theory. Springer Verlag, New York, 1995.

    Book  MATH  Google Scholar 

  • Vapnik V. Statistical Learning Theory. Wiley, NY, 1998.

    MATH  Google Scholar 

  • Vapnik V. The Nature of Statistical Learning Theory. Springer Science. 2013.

    MATH  Google Scholar 

  • Vapnik V. and Chapelle O. Bounds on error expectation for support vector machines. Neural Computation 2000; 12(9):2013–2036.

    Article  Google Scholar 

  • Wang, D.I.: Online Support Vector Machine Based on Convex Hull Vertices Selection. IEEE Transactions on Neural Networks Learning Systems 2013; 24(4), 593–608.

    Article  MathSciNet  Google Scholar 

  • Weston J. and Herbrich R.. Adaptive margin support vector machines. In Smola A.J., Bartlett P.L., Scholkopf B., and Schuurmans D., Editors, Advances in Large Margin Classifiers, pages 281–296, MIT Press, Cambridge, MA, 2000,

    Google Scholar 

  • Wenzel F. Galy-Fajou T. Deutsch M., Kloft M. Bayesian Nonlinear Support Vector Machines for Big Data. ECML/PKDD 2017: 307–322

    Google Scholar 

  • Williamson R. C., Smola A. J., and Scholkopf B. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory 2001; 47(6):2516–2532.

    Article  MathSciNet  MATH  Google Scholar 

  • Wolfe P. A duality theorem for non-linear programming. Quarterly of Applied Mathematics 1961; 19:239–244.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhou X., Zhang X. and Wang B., Online Support Vector Machine: A Survey, in Kim J.H and Geem Z. W. editors, Harmony Search Algorithm, pages 269–278, Proceedings of the 2nd International Conference on Harmony Search Algorithm (ICHSA 2015).

    Google Scholar 

  • Zien A., Ratsch G., Mika S., Scholkopf B., Lengauer T. and Muller K.R. Engineering support vector machine kernels that recognize translation initiation sites. Bio-Informatics 2000; 16(9):799–807.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ben-Gurion University, Beersheba, Israel

    Armin Shmilovici

Authors
  1. Armin Shmilovici
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Armin Shmilovici .

Editor information

Editors and Affiliations

  1. Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Lior Rokach

  2. Department of Industrial Engineering, Tel Aviv University, Ramat Aviv, Israel

    Oded Maimon

  3. Department of Industrial Engineering, Tel Aviv University, Tel Aviv, Israel

    Erez Shmueli

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Shmilovici, A. (2023). Support Vector Machines. In: Rokach, L., Maimon, O., Shmueli, E. (eds) Machine Learning for Data Science Handbook. Springer, Cham. https://doi.org/10.1007/978-3-031-24628-9_6

Download citation

Publish with us

Policies and ethics

Search

Navigation