Abstract
We reduce the classification problem to solving a global optimization problem and a method based on a combination of the cutting angle method and a local search is applied to the solution of this problem. The proposed method allows to solve classification problems for databases with an arbitrary number of classes. Numerical experiments have been carried out with databases of small to medium size. We present their results and provide comparisons of these results with those obtained by 29 different classification algorithms. The best performance overall was achieved with the global optimization method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. Abello, P. M. Pardalos, and M. G. C. Resende (eds.), Handbook of Massive Datasets, Kluwer Academic Publishers: Dordrecht, 2001.
M. Yu. Andramonov, A. M. Rubinov, and B. M. Glover, “Cutting angle method in global optimization,” Applied Mathematics Letters vol. 12, PP. 95–100, 1999.
A. M. Bagirov, “A method of approximating a subdifferential,” Russian Journal of Computational Mathematics and Mathematical Physics vol. 32, pp. 561–566, 1992.
A. M. Bagirov, “Continuous subdifferential approximation and its construction,” Indian Journal of Pure and Applied Mathematics vol. 1, pp. 17–29, 1998.
A. M. Bagirov, “Derivative-free methods for unconstrained nonsmooth optimization and its numerical analysis” Investigacao Operacional vol. 19, pp. 75–93, 1999a.
A. M. Bagirov, “Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices,” in Progress in Optimization: Contribution from Australasia A. Eberhard et al., eds. Kluwer Academic Publishers, pp. 147–175, 1999b.
A. M. Bagirov, “Numerical methods for minimizing quasidifferentiable functions: A survey and comparison,” in Quasidifferentiability and Related Topics, V. F. Demyanov and A. M. Rubinov, eds. Kluwer Academic Publishers, pp. 33–71, 2000.
A. M. Bagirov and A. A. Gasanov, “A method of approximating a quasidifferential,” Russian Journal of Computational Mathematics and Mathematical Physics vol. 35, pp. 403–409, 1995.
A. M. Bagirov and A. M. Rubinov, “Global minimization of increasing positively homogeneous function over unit simplex,” Annals of Operations Research vol. 98, pp. 171–187, 2000.
A. M. Bagirov and A. M. Rubinov, “Modified versions of the cutting angle method, In: Advances in Convex Analysis and Global Optimization. N. Hadjisavvas and P. M. Pardalos (eds), Kluwer Academic Publishers, Dordrecht, pp. 245–268, 2001.
A. M. Bagirov and A. M. Rubinov, “The cutting angle method and a local search,” Journal of Global Optimization to appear.
A. M. Bagirov, A. M. Rubinov, and J. Yearwood, “Using global optimization to improve classification for medical diagnosis and prognosis,” Topics in Health Information Management. vol. 22, pp. 65–74, 2001.
A. M. Bagirov, A. M. Rubinov, and J. Yearwood, “A heuristic algorithm for feature selection based on optimization techniques,” in Heuristic and Optimization for Knowledge Discovery, R. Sarker, H. Abbas, and C. S. Newton, eds., Idea Publishing Group, 2002.
K. P. Bennett and O. L. Mangasarian, “Robust linear programming discrimination of two linearly inseparable sets,” Optimization Methods and Software vol. 1, pp. 23–34, 1992.
P. S. Bradley, U. M. Fayyad, and O. L. Mangasarian, “Data mining: Overview and optimization opportunities,” INFORMS Journal on Computing vol. 11, pp. 217–238, 1999.
P. S. Bradley and O. L. Mangasarian, “Feature selection via concave minimization and support vector machines,” in Machine Learning Proceedings of the Fifteenth International Conference (ICML'98), J. Shavlik, ed., Morgan Kaufmann: San Francisco, California, pp. 82–90, 1998.
P. S. Bradley and O. L. Mangasarian, “Massive data discrimination via linear support vector machines,” Optimization Methods and Software vol. 13, pp. 1–10, 2000.
C. Chen and O. L. Mangasarian, “Hybrid misclassification minimization,” Mathematical Programming Technical Report, 95–05, University of Wisconsin, 1995.
V. F. Demyanov and A. M. Rubinov, Constructive Nonsmooth Analysis, Peter Lang: Frankfurt am Main, 1995.
R. Dubes and A. K. Jain, “Clustering techniques: The user's dilemma,” Pattern Recognition vol. 8, pp. 247–260, 1976.
K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edn., Academic Press, 1990.
D. M. Houkins, M.W. Muller, and J. A. ten Krooden, “Cluster analysis,” in Topics in Applied Multivariate Analysis, Cambridge University press: Cambridge, 1982.
O. L. Mangasarian, “Misclassification minimization,” Journal of Global Optimization vol. 5, pp. 309–323, 1994.
O. L. Mangasarian, “Mathematical programming in data mining,” Data Mining and Knowledge Discovery vol. 1, pp. 183–201, 1997.
G. J. McLachlan, Discriminat Analysis and Statistical Pattern Recognition, John Wiley: New York, 1992.
D. Michie, D. J. Spiegelhalter, and C. C. Taylor (eds.), “Machine Learning, Neural and Statistical Classification,” In Ellis Horwood Series in Artificial Intelligence, London, 1994.
R. Mifflin, “Semismooth and semiconvex functions in constrained optimization,” SIAM Journal of Control and Optimization vol. 15, pp. 957–972, 1977.
P. M. Murphy and D. W. Aha, “UCI repository of machine learning databases,” Technical report, Department of Information and Computer science, University of California, Irvine, 1992. www.ics.uci.edu/mlearn/ MLRepository.html.
J. R. Quinlan, C4.5. Programs for Machine Learning, Morgan Kaufmann: San Mateo, 1992.
J. R. Quinlan, C4.5. Programs for Machine Learning, Morgan Kaufmann: San Mateo, 1993.
A. M. Rubinov, Abstract Convexity and Global Optimization, Kluwer Academic Publishers: Dordrecht, 2000.
A. M. Rubinov, N. V. Soukhoroukova, and J. Yearwood, “Clustering for studying structure and quality of datasets,” Research Report 01/24, University of Ballarat, 2001.
H. Spath, Cluster Analysis Algorithms, Ellis Horwood Limited: Chichester, 1980.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bagirov, A.M., Rubinov, A.M. & Yearwood, J. A Global Optimization Approach to Classification. Optimization and Engineering 3, 129–155 (2002). https://doi.org/10.1023/A:1020911318981
Issue Date:
DOI: https://doi.org/10.1023/A:1020911318981