Empirical risk minimization and problems of constructing linear classifiers
- 41 Downloads
Problems of construction of linear classifiers for classifying many sets are considered. In the case of linearly separable sets, problem statements are given that generalize already well-known formulations. For linearly inseparable sets, a natural criterion for choosing a classifier is empirical risk minimization. A mixed integer formulation of the empirical risk minimization problem and possible solutions of its continuous relaxation are considered. The proposed continuous relaxation problem is compared with problems solved with the help of other approaches to the construction of linear classifiers. Features of nonsmooth optimization methods used to solve the formulated problems are described.
Keywordslinear classification algorithm linear separability of sets empirical risk support vector method optimization method
- 1.Yu. I. Zhuravlev, “An algebraic approach to recognition or classification problems,” Pattern Recognition and Image Analysis, 8, No. 1, 59–100 (1998).Google Scholar
- 2.L. M. Mestetskii, Mathematical Methods for Pattern Recognition [in Russian], www.intuit.ru/department/graphics/imageproc/.
- 3.K. V. Vorontsov, Machine Learning [in Russian], www.machinelearning.ru/wiki/images/6/68/voron-ML-Lin.pdf.
- 4.A. M. Gupal and I. V. Sergienko, Optimal Pattern Recognition Procedures [in Russian], Naukova Dumka, Kyiv (2008).Google Scholar
- 5.M. Shlezinger and V. Glavach, Ten Lectures on Statistical and Structural Recognition [in Russian], Naukova Dumka, Kyiv (2004).Google Scholar
- 6.Yu. Laptin and A. Vinogradov, “Exact discriminant function design using some optimization techniques,” in: Classification, Forecasting, Data Mining: International Book Series “INFORMATION SCIENCE & COMPUTING”, No. 8, Sofia, Bulgaria (2009), pp. 14–19.Google Scholar
- 9.B. V. Rublev, Yu. I. Petunin, and P.G. Litvinko, “Structure of homothetic linearly separable sets in an n-dimensional Euclidean space,” Cybernetics and Systems Analysis, Vol. 28, Part 1, No. 1, 1–10 (1992) and Part 2, No. 2, 180–188 (1992).Google Scholar
- 10.P. I. Stetsyuk, O. A. Berezovsky, M.G. Zhurbenko, and D. O. Kropotov, Methods of Nonsmooth Optimization in Special Classification Problems [in Ukrainian], Prepr. 2009–1, V. M. Glushkov Institute of Cybernetics of NAS of Ukraine, Kyiv (2009).Google Scholar
- 11.K. P. Bennett and O. L. Mangasarian, “Robust linear programming discrimination of two linearly inseparable sets,” Optimiz. Methods and Software, No. 5, 23–34 (1992).Google Scholar
- 12.N. G. Zhurbenko and D. Kh. Saimbetov, “To the numerical solution of one class of problems of robust partitioning for two sets,” in: Methods for the Investigation of Extremal Problems, V. M. Glushkov Cybernetics Institute of NAN of Ukraine (1994), pp. 52–55.Google Scholar
- 13.N. Z. Shor, Nondifferentiable Optimization and Polynomial Problems, Kluwer (1998).Google Scholar
- 15.Yu. P. Laptin and A. P. Likhovid, “The use of convex extensions of functions to solve nonlinear optimization problems,” USiM, No. 6, 25–31 (2010).Google Scholar