Skip to main content
SpringerLink
Log in
Book cover

Data Mining and Knowledge Discovery Handbook pp 231–247Cite as

Support Vector Machines

  • Chapter
  • First Online:

Summary

Support Vector Machines (SVMs) are a set of related methods for supervised learning, applicable to both classification and regression problems. A SVM classifiers creates a maximum-margin hyperplane that lies in a transformed input space and splits the example classes, while maximizing the distance to the nearest cleanly split examples. The parameters of the solution hyperplane are derived from a quadratic programming optimization problem. Here, we provide several formulations, and discuss some key concepts.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   349.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Bazaraa M. S., Sherali H. D., and Shetty C. M. Nonlinear programming: theory and algorithms. Wiley, second edition, 1993.

    Google Scholar 

  • Bertsekas D.P. Nonlinear Programming. Athena Scientific, MA, 1995.

    MATH  Google Scholar 

  • Chang C.-C. and Lin C.-J. Training support vector classifiers: Theory and algorithms. Neural Computation 2001; 13(9):2119–2147.

    Article  MATH  Google Scholar 

  • Chang C.-C. and Lin C.-J. (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm.

  • Chen P.-H., Lin C. -J., and Scholkopf B. A tutorial on nu-support vector machines. 2003.

    Google Scholar 

  • Chew H. G., Lim C. C., and Bogner R. E. An implementation of training dual-nu support vector machines. In Qi, Teo, and Yang, editors, Optimization and Control with Applications. Kluwer, 2003.

    Google Scholar 

  • Chu W. Bayesian approach to support vector machines. PhD thesis, National University of Singapore , 2003; Available online http://citeseer.ist.psu.edu/chu03bayesian.html

  • Chung K.-M., Kao W.-C., Sun C.-L., and Lin C.-J. Decomposition methods for linear support vector machines. Neural Computation 2004; 16(8):1689-1704).

    Article  MATH  Google Scholar 

  • Cortes C. and Vapnik V. Support vector networks. Machine Learning 1995; 20:273–297.

    MATH  Google Scholar 

  • Cristianini N. and Shawe-Taylor J. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge Univ. Press, 2000.

    Google Scholar 

  • Dumais S. Using SVMs for text categorization. IEEE Intelligent Systems 1998; 13(4).

    Google Scholar 

  • Hsu C.-W. and Lin C.-J. A comparison of methods for multi-class support vector machines IEEE Transactions on Neural Networks 2002; 13(2); 415–425.

    Article  Google Scholar 

  • Hsu C.-W. Chang C.-C and Lin C.-J. A practical guide to support vector classification. 2003. Available Online: www.csie.ntu.edu.tw/∼cjlin/papers/guide/guide.pdf

  • Isabelle 2004, (a collection of SVM applications) Available Online: http://www.clopinet.com/isabelle/Projects/SVWM/applist.html

  • Joachims T. Making large–scale SVM learning practical. In Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods — Support Vector Learning, pages 169–184, Cambridge, MA, MIT Press, 1999.

    Google Scholar 

  • Joachims T. Learning to Classify Text using Support Vector Machines Methods, Theory, and Algorithms. Kluwer Academic Publishers, 2002.

    Google Scholar 

  • Joachims T. 2004, SVMlight, available online http://www.cs.cornell.edu/People/tj/svmlight/

  • Kernel 2004, (a collection of literature, software and Web pointers dealing with SVM and Gaussian processes) Available Online http://www.kernel-machines.org.

  • Law M. H. and Kwok J. T. Bayesian support vector regression. Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics (AISTATS) pages 239-244, Key-West, Florida, USA, January 2000.

    Google Scholar 

  • Lin C.-J. Formulations of support vector machines: a note from an optimization point of view. Neural Computation 2001; 13(2):307–317.

    Article  MATH  Google Scholar 

  • Lin C.-J. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks 2001; 12(6):1288–1298.

    Article  Google Scholar 

  • Martin D. R., Fowlkes C. C., and Malik J. Learning to detect natural image boundaries using brightness and texture. In Advances in Neural Information Processing Systems, volume 14, 2002.

    Google Scholar 

  • Mukherjee S., Osuna E., and Girosi F. Nonlinear prediction of chaotic time series using a support vector machine. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 511–520, New-York, IEEE Press, 1997.

    Chapter  Google Scholar 

  • Muller K.-R., Mika S., Ratsch G., Tsuda K., and Scholkopf B., An introduction to kernel-based learning algorithms. IEEE Neural Networks 2001; 12(2):181-201.

    Article  Google Scholar 

  • Osuna E., Freund R., and Girosi F. An improved training algorithm for support vector machines. In Principe J., Gile L., Morgan N. and Wilson E. editors, Neural Networks for Signal Processing VII - proceedings of the 1997 IEEE Workshop, pages 276-285, New-York, IEEE Press, 1997.

    Chapter  Google Scholar 

  • Platt J. C. Fast training of support vector machines using sequential minimal optimization. In Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods - Support Vector Learning, Cambridge, MA, MIT Press, 1998.

    Google Scholar 

  • Ratsch G., Onoda T., and Muller K.R. Soft margins for AdaBoost. Machine Learning 2001; 42(3):287–320.

    Article  Google Scholar 

  • Rifkin R. and Klautau A.. In Defense of One-vs-All Classification, Journal of Machine Learning Research 2004; 5:101-141.

    MathSciNet  Google Scholar 

  • Scholkopf B., Support Vector Learning. Oldenbourg Verlag, Munich, 1997.

    Google Scholar 

  • Scholkopf B., Statistical learning and kernel methods, Technical Report MSRTR-2000-23, Available Online http://research.microsoft.com/research/pubs/view.aspx?msr tr id= MSR-TR-2000-23

  • Scholkopf B., Burges C.J.C., and Vapnik V.N. Extracting support data for a given task. In Fayyad U.M. and Uthurusamy R., Editors, Proceedings, First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, CA, 1995.

    Google Scholar 

  • Scholkopf B., Simard P.Y., Smola A.J., and Vapnik V.N.. Prior knowledge in support vector kernels. In Jordan M., Kearns M., and Solla S., Editors, Advances in Neural Information Processing Systems 10, pages 640–646. MIT Press, Cambridge, MA, 1998.

    Google Scholar 

  • Scholkopf B., Burges C. J. C., and Smola A. J., editors, Advances in Kernel Methods - Support Vector Learning, Cambridge, MA, MIT Press, 1999.

    Google Scholar 

  • Scholkopf B. and Smola A. J. Learning with Kernels. MIT Press, Cambridge, MA, 2002.

    Google Scholar 

  • Scholkopf B., Smola A. J., Williamson R. C., and Bartlett P. L. New support vector algorithms. Neural Computation 2000; 12:1207–1245.

    Article  Google Scholar 

  • Shawe-Taylor J. and Cristianini N. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.

    Google Scholar 

  • Smola A. J., Bartlett P. L., Scholkopf B. and Schuurmans D. Advances in Large Margin Classifiers. MIT Press, Cambridge, MA, 2000.

    MATH  Google Scholar 

  • Smola A.J. and Scholkopf B.. A tutorial on support vector regression. Statistics and Computing 2004; 14(13):199-222.

    Article  MathSciNet  Google Scholar 

  • Smola A.J., Scholkopf B. and Ratsch G. Linear programs for automatic accuracy control in regression. Proceedings of International Conference on Artificial Neural Networks ICANN’99, Berlin, Springer 1999.

    Google Scholar 

  • Steinwart I. On the optimal parameter choice for nu-support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003; 25: 1274-1284.

    Article  Google Scholar 

  • Steinwart I. Sparseness of support vector machines. Journal of Machine Learning Research 2004; 4(6):1071-1105.

    Article  MATH  MathSciNet  Google Scholar 

  • Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least Squares Support Vector Machines. World Scientific Publishing, Singapore, 2002.

    Book  MATH  Google Scholar 

  • Vapnik V. The Nature of Statistical Learning Theory . Springer Verlag, New York, 1995.

    MATH  Google Scholar 

  • Vapnik V. Statistical Learning Theory. Wiley, NY, 1998.

    MATH  Google Scholar 

  • Vapnik V. and Chapelle O. Bounds on error expectation for support vector machines. Neural Computation 2000; 12(9):2013–2036.

    Article  Google Scholar 

  • Weston J. and Herbrich R., Adaptive margin support vector machines. In Smola A.J., Bartlett P.L., Scholkopf B., and Schuurmans D., Editors, Advances in Large Margin Classifiers, pages 281–296, MIT Press, Cambridge, MA, 2000,.

    Google Scholar 

  • Williamson R. C., Smola A. J., and Scholkopf B., Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory 2001; 47(6):2516–2532.

    Article  MATH  MathSciNet  Google Scholar 

  • Wolfe P. A duality theorem for non-linear programming. Quartely of Applied Mathematics 1961; 19:239–244.

    MATH  MathSciNet  Google Scholar 

  • Zien A., Ratsch G., Mika S., Scholkopf B., Lengauer T. and Muller K.R. Engineering support vector machine kernels that recognize translation initiation sites. Bio-Informatics 248 Armin Shmilovici 16(9):799–807.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ben-Gurion University, Beer-Sheva, Israel

    Armin Shmilovici

Authors
  1. Armin Shmilovici
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. , Dept. Industrial Engineering, Tel Aviv University, Ramat Aviv, 69978, Israel

    Oded Maimon

  2. , Dept. Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, 84105, Israel

    Lior Rokach

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Shmilovici, A. (2009). Support Vector Machines. In: Maimon, O., Rokach, L. (eds) Data Mining and Knowledge Discovery Handbook. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-09823-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-0-387-09823-4_12

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-387-09822-7

  • Online ISBN: 978-0-387-09823-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation