Annals of Operations Research

, Volume 74, Issue 0, pp 65–88 | Cite as

Constrained discriminant analysis via 0/1 mixed integer programming

  • Richard J. Gallagher
  • Eva K. Lee
  • David A. Patterson


A nonlinear 0/1 mixed integer programming model is presented for a constrained discriminant analysis problem. The model places restrictions on the numbers of misclassifications allowed among the training entities, and incorporates a "reserved judgment" region to which entities whose classifications are difficult to determine may be allocated. Two linearizations of the model are given one heuristic and one exact. Numerical results from real-world machine-learning datasets are presented.


Discriminant Analysis Linear Discriminant Analysis Problem Instance Mixed Integer Programming Mixed Integer Programming Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    P.L. Abad and W.J. Banks, New LP based heuristics for the classification problem, European Journal of Operational Research 67(1993)88–100.CrossRefGoogle Scholar
  2. [2]
    J.A. Anderson, Constrained discrimination between k populations, Journal of the Royal Statistical Society, Series B 31(1969)123–139.Google Scholar
  3. [3]
    T.W. Anderson, An Introduction to Multivariate Statistical Analysis, 2nd ed., Wiley, New York, 1984.Google Scholar
  4. [4]
    D. Applegate, R.E. Bixby, V. Chvátal and W. Cook, The traveling salesman problem, Technical Report, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 1994.Google Scholar
  5. [5]
    S.M. Bajgier and A.V. Hill, An experimental comparison of statistical and linear programming approaches to the discriminant problem, Decision Sciences 13(1982)604–618.Google Scholar
  6. [6]
    W.J. Banks and P.L. Abad, An efficient optimal solution algorithm for the classification problem, Decision Sciences 22(1991)1008–1023.Google Scholar
  7. [7]
    K.P. Bennett and O.L. Mangasarian, Multicategory discrimination via linear programming, Optimization Methods and Software 3(1993)27–39.Google Scholar
  8. [8]
    R.E. Bixby, W. Cook, A. Cox and E.K. Lee, Computational experience with parallel mixed integer programming in a distributed environment, Research Monograph CRPC-TR95554, Center for Research on Parallel Computation, Rice University, Houston, TX, 1995.Google Scholar
  9. [9]
    J.D. Broffit, R.H. Randles and R.V. Hogg, Distribution-free partial discriminant analysis, Journal of the American Statistical Association 71(1976)934–939.CrossRefGoogle Scholar
  10. [10]
    T.M. Cavalier, J.P. Ignizio and A.L. Soyster, Discriminant analysis via mathematical programming: Certain problems and their causes, Computers and Operations Research 16(1989)353–362.CrossRefGoogle Scholar
  11. [11]
    CPLEX 3.0, CPLEX Optimization, Inc., Incline Village, NV.Google Scholar
  12. [12]
    A.P. Duarte Silva and A. Stam, Second order mathematical programming formulations for discriminant analysis, European Journal of Operational Research 72(1994)4–22.CrossRefGoogle Scholar
  13. [13]
    R.A. Fisher, The use of multiple measurements in taxonomic problems, Annals of Eugenics 7(1936)179–188.Google Scholar
  14. [14]
    N. Freed and F. Glover, A linear programming approach to the discriminant problem, Decision Sciences 12(1981)68–74.Google Scholar
  15. [15]
    N. Freed and F. Glover, Evaluating alternative linear programming models to solve the two-group discriminant problem, Decision Sciences 17(1986)151–162.Google Scholar
  16. [16]
    M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, New York, 1979.Google Scholar
  17. [17]
    M.P. Gessaman and P.H. Gessaman, A comparison of some multivariate discrimination procedures, Journal of the American Statistical Association 67(1972)468–472.CrossRefGoogle Scholar
  18. [18]
    W.V. Gehrlein, General mathematical programming formulations for the statistical classification problem, Operations Research Letters 5 (1986)299–304.CrossRefGoogle Scholar
  19. [19]
    F. Glover, Improved linear programming models for discriminant analysis, Decision Sciences 21(1990)771–785.Google Scholar
  20. [20]
    F. Glover, S. Keene and B. Duea, A new class of models for the discriminant problem, Decision Sciences 19(1988)269–280.Google Scholar
  21. [21]
    J.D.F. Habbema, J. Hermans and A.T. Van Der Burgt, Cases of doubt in allocation problems, Biometrika 61(1974)313–324.CrossRefGoogle Scholar
  22. [22]
    G.J. Koehler and S.S. Erenguc, Minimizing misclassifications in linear discriminant analysis, Decision Sciences 21(1990)63–85.Google Scholar
  23. [23]
    E.K. Lee and J.E. Mitchell, An interior-point based branch-and-bound solver for nonlinear mixed integer programs, Working paper, Department of Industrial Engineering and Operations Research, Columbia University, New York, NY, 1996.Google Scholar
  24. [24]
    J.M. Liittschwager and C. Wang, Integer programming solution of a classification problem, Management Science 24(1978)1515–1525.CrossRefGoogle Scholar
  25. [25]
    O.L. Mangasarian, W.N. Street and W.H. Wolberg, Breast cancer diagnosis and prognosis via linear programming, Operations Research 43(1995)570–577.Google Scholar
  26. [26]
    G.J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition, Wiley, New York, 1992.CrossRefGoogle Scholar
  27. [27]
    P.M. Murphy and D.W. Aha, UCI Repository of machine learning databases [], Department of Information and Computer Science, University of California, Irvine, CA, 1994.Google Scholar
  28. [28]
    R. Nath, W.M. Jackson and T.W. Jones, A comparison of the classical and the linear programming approaches to the classification problem in discriminant analysis, Journal of Statistical Computation and Simulation 41(1992)73–93.Google Scholar
  29. [29]
    G.L. Nemhauser and L.A. Wolsey, Integer and Combinatorial Optimization, Wiley, New York, 1988.Google Scholar
  30. [30]
    T-H. Ng and R.H. Randles, Distribution-free partial discrimination procedures, Computers and Mathematics with Applications 12A(1986)225–234.CrossRefGoogle Scholar
  31. [31]
    D.A. Patterson, Three population constrained discrimination, Working paper, Department of Mathematical Sciences, University of Montana, Missoula, MT, 1996.Google Scholar
  32. [32]
    C.P. Quesenberry and M.P. Gessaman, Nonparametric discrimination using tolerance regions, Annals of Mathematical Statistics 39(1968)664–673.Google Scholar
  33. [33]
    A. Stam and E.A. Joachimsthaler, Solving the classification problem in discriminant analysis via linear and nonlinear programming, Decision Sciences 20(1989)285–293.Google Scholar
  34. [34]
    A. Stam and E.A. Joachimsthaler, A comparison of a robust mixed-integer approach to existing methods for establishing classification rules for the discriminant problem, European Journal of Operational Research 46(1990)113–122.CrossRefGoogle Scholar
  35. [35]
    A. Stam and C.T. Ragsdale, On the classification gap in mathematical-programming-based approaches to the discriminant problem, Naval Research Logistics 39(1992)545–559.Google Scholar
  36. [36]
    G. Wagner, P. Tautu and U. Wolbler, Problems of medical diagnosis — a bibliography, Methods of Information in Medicine 17(1978)55–74.Google Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Richard J. Gallagher
  • Eva K. Lee
  • David A. Patterson

There are no affiliations available

Personalised recommendations