Empirical risk minimization and problems of constructing linear classifiers

  • Y. P. LaptinEmail author
  • Y. I. Zhuravlev
  • A. P. Vinogradov


Problems of construction of linear classifiers for classifying many sets are considered. In the case of linearly separable sets, problem statements are given that generalize already well-known formulations. For linearly inseparable sets, a natural criterion for choosing a classifier is empirical risk minimization. A mixed integer formulation of the empirical risk minimization problem and possible solutions of its continuous relaxation are considered. The proposed continuous relaxation problem is compared with problems solved with the help of other approaches to the construction of linear classifiers. Features of nonsmooth optimization methods used to solve the formulated problems are described.


linear classification algorithm linear separability of sets empirical risk support vector method optimization method 


  1. 1.
    Yu. I. Zhuravlev, “An algebraic approach to recognition or classification problems,” Pattern Recognition and Image Analysis, 8, No. 1, 59–100 (1998).Google Scholar
  2. 2.
    L. M. Mestetskii, Mathematical Methods for Pattern Recognition [in Russian],
  3. 3.
    K. V. Vorontsov, Machine Learning [in Russian],
  4. 4.
    A. M. Gupal and I. V. Sergienko, Optimal Pattern Recognition Procedures [in Russian], Naukova Dumka, Kyiv (2008).Google Scholar
  5. 5.
    M. Shlezinger and V. Glavach, Ten Lectures on Statistical and Structural Recognition [in Russian], Naukova Dumka, Kyiv (2004).Google Scholar
  6. 6.
    Yu. Laptin and A. Vinogradov, “Exact discriminant function design using some optimization techniques,” in: Classification, Forecasting, Data Mining: International Book Series “INFORMATION SCIENCE & COMPUTING”, No. 8, Sofia, Bulgaria (2009), pp. 14–19.Google Scholar
  7. 7.
    Yu. P. Laptin, A. P. Likhovid, and A. P. Vinogradov, “Approaches to construction of linear classifiers in the case of many classes,” Pattern Recognition and Image Analysis 20, No. 2, 137–145 (2010).CrossRefGoogle Scholar
  8. 8.
    Yu. I. Petunin and G. A. Shul’deshov, “Pattern recognition with Fisher linear discriminant functions,” Cybernetics, Vol. 15, No. 6, 925–928 (1979).MathSciNetGoogle Scholar
  9. 9.
    B. V. Rublev, Yu. I. Petunin, and P.G. Litvinko, “Structure of homothetic linearly separable sets in an n-dimensional Euclidean space,” Cybernetics and Systems Analysis, Vol. 28, Part 1, No. 1, 1–10 (1992) and Part 2, No. 2, 180–188 (1992).Google Scholar
  10. 10.
    P. I. Stetsyuk, O. A. Berezovsky, M.G. Zhurbenko, and D. O. Kropotov, Methods of Nonsmooth Optimization in Special Classification Problems [in Ukrainian], Prepr. 2009–1, V. M. Glushkov Institute of Cybernetics of NAS of Ukraine, Kyiv (2009).Google Scholar
  11. 11.
    K. P. Bennett and O. L. Mangasarian, “Robust linear programming discrimination of two linearly inseparable sets,” Optimiz. Methods and Software, No. 5, 23–34 (1992).Google Scholar
  12. 12.
    N. G. Zhurbenko and D. Kh. Saimbetov, “To the numerical solution of one class of problems of robust partitioning for two sets,” in: Methods for the Investigation of Extremal Problems, V. M. Glushkov Cybernetics Institute of NAN of Ukraine (1994), pp. 52–55.Google Scholar
  13. 13.
    N. Z. Shor, Nondifferentiable Optimization and Polynomial Problems, Kluwer (1998).Google Scholar
  14. 14.
    Yu. P. Laptin, “An approach to the solution of nonlinear constrained optimization problems,” Cybernetics and Systems Analysis, Vol. 45, No. 3, 497–502 (2009).MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Yu. P. Laptin and A. P. Likhovid, “The use of convex extensions of functions to solve nonlinear optimization problems,” USiM, No. 6, 25–31 (2010).Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2011

Authors and Affiliations

  • Y. P. Laptin
    • 1
    Email author
  • Y. I. Zhuravlev
    • 2
  • A. P. Vinogradov
    • 2
  1. 1.V. M. Glushkov Institute of Cybernetics, National Academy of Sciences of UkraineKyivUkraine
  2. 2.A. A. Dorodnitsyn Computing CenterRussian Academy of SciencesMoscowRussia

Personalised recommendations