Skip to main content
SpringerLink
Log in

Empirical risk minimization and problems of constructing linear classifiers

  • Published:
Cybernetics and Systems Analysis Aims and scope

Abstract

Problems of construction of linear classifiers for classifying many sets are considered. In the case of linearly separable sets, problem statements are given that generalize already well-known formulations. For linearly inseparable sets, a natural criterion for choosing a classifier is empirical risk minimization. A mixed integer formulation of the empirical risk minimization problem and possible solutions of its continuous relaxation are considered. The proposed continuous relaxation problem is compared with problems solved with the help of other approaches to the construction of linear classifiers. Features of nonsmooth optimization methods used to solve the formulated problems are described.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Yu. I. Zhuravlev, “An algebraic approach to recognition or classification problems,” Pattern Recognition and Image Analysis, 8, No. 1, 59–100 (1998).

    Google Scholar 

  2. L. M. Mestetskii, Mathematical Methods for Pattern Recognition [in Russian], www.intuit.ru/department/graphics/imageproc/.

  3. K. V. Vorontsov, Machine Learning [in Russian], www.machinelearning.ru/wiki/images/6/68/voron-ML-Lin.pdf.

  4. A. M. Gupal and I. V. Sergienko, Optimal Pattern Recognition Procedures [in Russian], Naukova Dumka, Kyiv (2008).

    Google Scholar 

  5. M. Shlezinger and V. Glavach, Ten Lectures on Statistical and Structural Recognition [in Russian], Naukova Dumka, Kyiv (2004).

    Google Scholar 

  6. Yu. Laptin and A. Vinogradov, “Exact discriminant function design using some optimization techniques,” in: Classification, Forecasting, Data Mining: International Book Series “INFORMATION SCIENCE & COMPUTING”, No. 8, Sofia, Bulgaria (2009), pp. 14–19.

  7. Yu. P. Laptin, A. P. Likhovid, and A. P. Vinogradov, “Approaches to construction of linear classifiers in the case of many classes,” Pattern Recognition and Image Analysis 20, No. 2, 137–145 (2010).

    Article  Google Scholar 

  8. Yu. I. Petunin and G. A. Shul’deshov, “Pattern recognition with Fisher linear discriminant functions,” Cybernetics, Vol. 15, No. 6, 925–928 (1979).

    MathSciNet  Google Scholar 

  9. B. V. Rublev, Yu. I. Petunin, and P.G. Litvinko, “Structure of homothetic linearly separable sets in an n-dimensional Euclidean space,” Cybernetics and Systems Analysis, Vol. 28, Part 1, No. 1, 1–10 (1992) and Part 2, No. 2, 180–188 (1992).

  10. P. I. Stetsyuk, O. A. Berezovsky, M.G. Zhurbenko, and D. O. Kropotov, Methods of Nonsmooth Optimization in Special Classification Problems [in Ukrainian], Prepr. 2009–1, V. M. Glushkov Institute of Cybernetics of NAS of Ukraine, Kyiv (2009).

  11. K. P. Bennett and O. L. Mangasarian, “Robust linear programming discrimination of two linearly inseparable sets,” Optimiz. Methods and Software, No. 5, 23–34 (1992).

  12. N. G. Zhurbenko and D. Kh. Saimbetov, “To the numerical solution of one class of problems of robust partitioning for two sets,” in: Methods for the Investigation of Extremal Problems, V. M. Glushkov Cybernetics Institute of NAN of Ukraine (1994), pp. 52–55.

  13. N. Z. Shor, Nondifferentiable Optimization and Polynomial Problems, Kluwer (1998).

  14. Yu. P. Laptin, “An approach to the solution of nonlinear constrained optimization problems,” Cybernetics and Systems Analysis, Vol. 45, No. 3, 497–502 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  15. Yu. P. Laptin and A. P. Likhovid, “The use of convex extensions of functions to solve nonlinear optimization problems,” USiM, No. 6, 25–31 (2010).

Download references

Author information

Authors and Affiliations

  1. V. M. Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine, Kyiv, Ukraine

    Y. P. Laptin

  2. A. A. Dorodnitsyn Computing Center, Russian Academy of Sciences, Moscow, Russia

    Y. I. Zhuravlev & A. P. Vinogradov

Authors
  1. Y. P. Laptin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. I. Zhuravlev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. P. Vinogradov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Y. P. Laptin.

Additional information

This work was carried out within the framework of the joint project No. 10-01-90419 of the National Academy of Sciences of Ukraine and Russian Foundation for Fundamental Investigations “Optimization approaches to problems of machine learning and data analysis.”

Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 155–164, July–August 2011.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Laptin, Y.P., Zhuravlev, Y.I. & Vinogradov, A.P. Empirical risk minimization and problems of constructing linear classifiers. Cybern Syst Anal 47, 640–648 (2011). https://doi.org/10.1007/s10559-011-9344-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10559-011-9344-0

Keywords

Search

Navigation