Skip to main content
SpringerLink
Log in

Massive Data Classification via Unconstrained Support Vector Machines

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

A highly accurate algorithm, based on support vector machines formulated as linear programs (Refs. 1–2), is proposed here as a completely unconstrained minimization problem (Ref. 3). Combined with a chunking procedure (Ref. 4), this approach, which requires nothing more complex than a linear equation solver, leads to a simple and accurate method for classifying million-point datasets. Because a 1-norm support vector machine underlies the proposed approach, the method suppresses input space features as well. A state-of-the-art linear programming package (CPLEX, Ref. 5) fails to solve problems handled by the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mangasarian, O. L., Generalized Support Vector Machines, Advances in Large Margin Classifiers, Edited by A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, MIT Press, Cambridge, Massachusetts, pp. 135–146, 2000; see website ftp://ftp.cs.wisc.edu/math-prog/tech-reports/98-14.ps.

    Google Scholar 

  2. Bradley, P. S., and Mangasarian, O. L., Feature Selection via Concave Minimization and Support Vector Machines, Machine Learning. Proceedings of the 15th International Conference (ICML '98), Edited by J. Shavlik, Morgan Kaufmann, San Francisco, California, pp. 82–90, 1998; see website ftp://ftp.cs.wisc.edu/math-prog/tech-reports/98-03.ps.

    Google Scholar 

  3. Mangasarian, O. L., Exact 1-Norm Support Vector Machines via Unconstrained Convex Differentiable Minimization, Report 05-03, Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin, 2005; see website ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/05-03.ps. Journal of Machine Learning Research, Vol 7, pp. 1517–1530, 2006.

  4. Bradley, P. S., and Mangasarian, O. L., Massive Data Discrimination via Linear Support Vector Machines, Optimization Methods and Software, Vol. 13, pp. 1–10, 2000; see website ftp://ftp.cs.wisc.edu/pub/math-prog/tech-reports/98-03.ps.

    MathSciNet  MATH  Google Scholar 

  5. ILOG, Incline Village, Nevada, ILOG CPLEX 9.0 User's Manual, 2003; see website http://www.ilog.com/products/cplex/.

  6. Cristianini, N., and Shawe-Taylor, J., An Introduction to Support Vector Machines, Cambridge University Press, Cambridge, Massachusetts, 2000.

    Google Scholar 

  7. Vapnik, V. N., The Nature of Statistical Learning Theory, 2nd Edition, Springer, New York, NY, 2000.

    MATH  Google Scholar 

  8. Schölkopf, B., and Smola, A., Learning with Kernels, MIT Press, Cambridge, Massachusetts, 2002.

    Google Scholar 

  9. Fung, G., and Mangasarian, O. L., A Feature Selection Newton Method for Support Vector Machine Classification, Computational Optimization and Applications, Vol. 28, pp. 185–202, 2004; see website ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/02-03.ps.

    Article  MathSciNet  MATH  Google Scholar 

  10. Hiriart-Urruty, J. B., Strodiot, J. J., and Nguyen, V. H., Generalized Hessian Matrix and Second-Order Optimality Conditions for Problems with C L1 Data, Applied Mathematics and Optimization, Vol. 11, pp. 43–56, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  11. Facchinei, F., Minimization of SC1 Functions and the Maratos Effect, Operations Research Letters, Vol. 17, pp. 131–137, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  12. Mangasarian, O. L., A Finite Newton Method for Classification Problems, Report 01–11, Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin, 2001; see website ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/01-11.ps. See paper in Optimization Methods and Software, Vol. 17, pp. 913–929, 2002.

  13. Mangasarian, O. L., Arbitrary-Norm Separating Plane, Operations Research Letters, Vol. 24, pp. 15–23, 1999; see website ftp://ftp.cs.wisc.edu/math-prog/tech-reports/97-07r.ps.

    Article  MathSciNet  MATH  Google Scholar 

  14. Matlab, User's Guide, The MathWorks, Natick, Massachusetts, 1994–2001; see website http://www.mathworks.com.

    Google Scholar 

  15. Gilmore, P. C., and Gomory, R. E., A Linear Programming Approach to the Cutting Stock Problem, Operations Research, Vol. 9, pp. 849–859, 1961.

    Article  MathSciNet  MATH  Google Scholar 

  16. Dantzig, G. B., and Wolfe, P., Decomposition Principle for Linear Programs, Operations Research, Vol. 8, pp. 101–111, 1960.

    MATH  Google Scholar 

  17. Chvátal, V., Linear Programming, W. H. Freeman and Company, New York, NY, 1983.

    MATH  Google Scholar 

  18. Murty, K, G., Linear Programming, John Wiley and Sons, New York, NY, 1983.

    MATH  Google Scholar 

  19. Luenberger, D. G., Linear and Nonlinear Programming, 2nd Edition, Addison-Wesley, Reading, Massachusetts, 1984.

    MATH  Google Scholar 

  20. Melli, G., Synthetic Classification Data Sets (SCDS), Report, School of Computing Science, Simon Fraser University, Burnaby, British Columbia, Canada, 1997; see websites http://fas.sfu.ca/cs/people/GradStudents/melli/SCDS/ and ftp://ftp.cs.wisc.edu/pub/dmi/data/SCDS/.

  21. Musicant, D. R., NDC: Normally Distributed Clustered Datasets, 1998; see website http://www.cs.wisc.edu/dmi/svm/ndc/.

  22. Murtagh, B. A., and Saunders, M. A., MINOS 5.0 User's Guide, Technical Report SOL 83.20, Stanford University, 1983; see version MINOS 5.4 Release Notes, 1992.

Download references

Author information

Authors and Affiliations

  1. Computer Sciences Department, University of Wisconsin, Madison, Wisconsin, USA

    O. L. Mangasarian (Professor Emeritus) & M. E. Thompson (Graduate Student)

Authors
  1. O. L. Mangasarian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. E. Thompson
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This research was supported by National Science Foundation Grants CCR-0138308 and IIS-0511905.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mangasarian, O.L., Thompson, M.E. Massive Data Classification via Unconstrained Support Vector Machines. J Optim Theory Appl 131, 315–325 (2006). https://doi.org/10.1007/s10957-006-9157-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-006-9157-x

Keywords

Search

Navigation