Separation of Data Via Concurrently Determined Discriminant Functions

  • Hong Seo Ryoo
  • Kwangsoo Kim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4484)


This paper presents a mixed 0 – 1 integer and linear programming (MILP) model for separation of data via a finite number of nonlinear and nonconvex discriminant functions. The MILP model concurrently optimizes the parameters of the user-provided individual discriminant functions and implements a decision boundary for an optimal separation of data under analysis.

The MILP model is extensively tested on six well-studied datasets in data mining research. The comparison of numerical results by the MILP-based classification of data with those produced by the multisurface method and the support vector machine in these experiments and the best from the literature illustrates the efficacy and the usefulness of the new MILP-based classification of data for supervised learning.


data classification machine learning mixed integer and linear programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, Y.J., Mangasarian, O., Wolberg, W.: Breast cancer survival and chemotherapy: A support vector machine analysis. In: Du, D., Pardalos, P., Wang, J. (eds.) Discrete Mathematical Problems with Medical Applications. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, vol. 55, pp. 1–20. American Mathematics Society (2000)Google Scholar
  2. 2.
    Mangasarian, O., Wolberg, W.: Cancer diagnosis via linear programming. SIAM Review 23(5), 1–18 (1990)Google Scholar
  3. 3.
    Carter, C., Catlett, S.: Assessing credit card applications using machine learning. IEEE Expert, 71–79 (1987)Google Scholar
  4. 4.
    Apté, C., Weiss, S., Grout, G.: Predicting defects in disk drive manufacturing: A case study in high-dimensional classification. In: Proceedings of the 9th Conference on Artificial Intelligence for Applications, Orlando, Florida, pp. 212–218 (1993)Google Scholar
  5. 5.
    Osuna, E., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 130–136 (1997)Google Scholar
  6. 6.
    Bhandari, I., et al.: Advanced scout: Data mining and knowledge discovery in nba. Data Mining and Knowledge Discovery 1, 121–125 (1997)CrossRefGoogle Scholar
  7. 7.
    Cortes, C., Vapnik, V.: Support vector networks. Machine Learning 20, 273–297 (1995)zbMATHGoogle Scholar
  8. 8.
    Megiddo, N.: On the complexity of polyhedral separability. Discrete and Computational Geometry 3, 325–337 (1988)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Bennett, K., Mangasarian, O.: Robust linear programming discrimination of two linearly inseparable sets. Optimization Methods and Software 1, 23–34 (1992)CrossRefGoogle Scholar
  10. 10.
    Falk, J., Lopez-Cardona, E.: The surgical separation of sets. Journal of Global Optimization 11, 433–462 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Bennett, K., Mangasarian, O.: Bilinear separation of two sets in n −space. Computational Optimization and Applications 2, 207–227 (1994)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Al-Khayyal, F., Falk, J.: Jointly constrained biconvex programming. Mathematics of Operations Research 8(2), 273–286 (1983)zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    Bennett, K.: Global tree optimization: A non-greedy decision tree algorithm. Computing Sciences and Statistics 26, 156–160 (1994)Google Scholar
  14. 14.
    Duda, R., Fossum, H.: Pattern classification by iteratively determined linear and piecewise linear discriminant functions. IEEE Transactions on Electronic Computers 15, 220–232 (1966)zbMATHCrossRefGoogle Scholar
  15. 15.
    Wolberg, W., Mangasarian, O.: Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences 87, 9193–9196 (1990)zbMATHCrossRefGoogle Scholar
  16. 16.
    ILOG CPLEX Division: CPLEX 9.0 User’s Manual, Incline, Nevada (2003)Google Scholar
  17. 17.
    Murphy, P., Aha, D.: Uci repository of machine learning databases: Readable data repository. Department of Computer Science, University of California at Irvine, CA (1994), Available from World Wide Web
  18. 18.
    Mangasarian, O.: Multisurface method of pattern separation. IEEE Transactions on Information Theory 14(6), 801–807 (1968)zbMATHCrossRefGoogle Scholar
  19. 19.
    Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Heidelberg (2000)zbMATHGoogle Scholar
  20. 20.
    Ryoo, H., Sahinidis, N.: Analysis of bounds for multilinear functions. Journal of Global Optimization 19(4), 403–424 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Mangasarian, O.: Generalized support vector machines. In: Smola, A., Bartlet, P., Schölkopf, B. (eds.) Advances in Large Margin Classifiers, pp. 135–146. MIT Press, Cambridge (2000)Google Scholar
  22. 22.
    Mangasarian, O., Musicant, D.: Data discrimination via nonlinear generalized support machines. In: Ferris, M., Mangasarian, O., Pang, J.S. (eds.) Complementarity: Applications, Algorithms and Extensions, Kluwer Academic Publishers, Dordrecht (2000)Google Scholar
  23. 23.
    Boros, E., et al.: An implementation of logical analysis of data. IEEE Transactions on Knowledge and Data Engineering 12, 292–306 (2000)CrossRefGoogle Scholar
  24. 24.
    Murthy, S., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. Journal of Artificial Intelligence Research 2, 1–32 (1994)zbMATHGoogle Scholar
  25. 25.
    Shavlik, J., Mooney, R., Towell, G.: Symbolic and neural learning algorithms: an experimental comparison. Machine Learning 6, 111–143 (1991)Google Scholar
  26. 26.
    Holte, R.: Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, 63–91 (1993)zbMATHCrossRefGoogle Scholar
  27. 27.
    Smith, J., et al.: Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In: Proceedings of the Twelfth Symposium on Computer Applications and Medical Care, pp. 261–265 (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Hong Seo Ryoo
    • 1
  • Kwangsoo Kim
    • 1
  1. 1.Division of Information Management Engineering, Korea University, 1, 5-Ka, Anam-Dong, Seongbuk-Ku, Seoul, 136-713Korea

Personalised recommendations