Skip to main content
SpringerLink
Log in

Hybrid misclassification minimization

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

Given two finite point setsA andB in then-dimensional real spaceR n, we consider the NP-complete problem of minimizing the number of misclassified points by a plane attempting to divideR n into two halfspaces such that each open halfspace contains points mostly ofA orB. This problem is equivalent to determining a plane {x | x T w=γ} that maximizes the number of pointsx ∈A satisfying inx T w>γ, plus the number of pointsx ∈B satisfyingx T w<γ. A simple but fast algorithm is proposed that alternates between (i) minimizing the number of misclassified points by translation of the separating plane, and (ii) a rotation of the plane so that it minimizes a weighted average sum of the distances of the misclassified points to the separating plane. Existence of a global solution to an underlying hybrid minimization problem is established. Computational comparison with a parametric approach to solve the NP-complete problem indicates that our approach is considerably faster and appears to generalize better as determined by tenfold cross-validation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. K.P. Bennett and E.J. Bredensteiner, A parametric optimization method for machine learning, Department of Mathematical Sciences Report No. 217, Rensselaer Polytechnic Institute, Troy, NY 12180, 1994.

    Google Scholar 

  2. K.P. Bennett and O.L. Mangasarian, Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software 1, 1992, 23–34.

    Google Scholar 

  3. R. Fourer, D. Gay and B. Kernighan,AMPL, The Scientific Press, South San Francisco, California, 1993.

    Google Scholar 

  4. M.R. Garey and D.S. Johnson,Computers and Intractability: A Guide to the Theory of NP-Completeness, W.H. Freeman and Company, San Francisco, 1979.

    Google Scholar 

  5. D. Heath,A Geometric Framework for Machine Learning, PhD thesis, Department of Computer Science, Johns Hopkins University-Baltimore, Maryland, 1992.

    Google Scholar 

  6. Z.-Q. Luo, J.-S. Pang, D. Ralph and S.-Q. Wu, Exact penalization and stationarity conditions of mathematical programs with equilibrium constraints, Technical Report 275, Communications Research Laboratory, McMaster University, Hamilton, Ontario-L8S 4K1, Canada, 1993; Mathematical Programming, to appear.

    Google Scholar 

  7. O.L. Mangasarian, Multi-surface method of pattern separation, IEEE Transactions on Information Theory IT-14, 1968, 801–807.

  8. O.L. Mangasarian, Misclassification minimization, Journal of Global Optimization 5, 1994, 309–323.

    Article  Google Scholar 

  9. O.L. Mangasarian, W.N. Street and W.H. Wolberg, Breast cancer diagnosis and prognosis via linear programming, Operations Research 43, 1995, 570–577.

    MathSciNet  Google Scholar 

  10. P.M. Murphy and D.W. Aha, UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, California, http://www.ics.uci.edu/AI/ML/MLDBRepository.html., 1992.

    Google Scholar 

  11. B.A. Murtagh and M.A. Saunders, MINOS 5.0 user's guide, Technical Report SOL 83.20, Stanford University, December 1983; MINOS 5.4 Release Notes, December 1992.

  12. S. Odewahn, E. Stockwell, R. Pennington, R. Humphreys and W. Zumach, Automated star/galaxy discrimination with neural networks, Astronomical Journal 103, 1992, 318–331.

    Article  Google Scholar 

  13. H.E. Scarf,The Computation of Economic Equilibria, Yale University Press, New Haven, Conneticut, 1973.

    Google Scholar 

  14. R.E. Steur,Multiple Criteria Optimization: Theory, Computation, and Application, Wiley, 1986.

  15. M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Statistical Society (Series B) 36, 1974, 111–147.

    Google Scholar 

  16. W.H. Wolberg, W.N. Street, D.N. Heisey and O.L. Mangasarian, Computer-derived nuclear grade and breast cancer prognosis, Analytical and Quantitative Cytology and Histology 17, 1995, 257–264.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Sciences Department, University of Wisconsin, 1210 West Dayton Street, 53706, Madison, WI, USA

    Chunhui Chen & O. L. Mangasarian

Authors
  1. Chunhui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. O. L. Mangasarian
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This material is based on research supported by Air Force Office of Scientific Research Grant F49620-94-1-0036 and National Science Foundation Grant CCR-9322479.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, C., Mangasarian, O.L. Hybrid misclassification minimization. Adv Comput Math 5, 127–136 (1996). https://doi.org/10.1007/BF02124738

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02124738

Keywords

Search

Navigation