Abstract
The Anderson discriminant function has a number of properties useful for solving classification problems and for evaluating posterior class probabilities. As the mathematical formalism, we use the same weighted least-squares method to approximate the Anderson discriminant function in the neighborhood of zero values both in solving the classification problem and in evaluating posterior probabilities of classes at a given point in the feature space. In the support vector method, the classification problem is solved by solving a quadratic programming problem with the number of constraints equal to the number of rows in the training sample, and to evaluate posterior probabilities of the classes an additional tool is used, namely the Platt calibrator, which converts the distance of the point to the boundary to the posterior probability of the class; the calibrator’s parameters are found with the maximum likelihood method. Using several examples of solving classification problems, we compare the performance of the methods by the criterion of empirical risk. The results are in favor of the method of approximating the Anderson discriminant function in the neighborhood of zero values.
Similar content being viewed by others
References
Anderson, T.W., An Introduction to Multivariate Statistical Analysis, New York: Wiley, 2003, 3rd ed.
Zenkov, V.V., Approximation of Discriminant Functions in the Neighborhood of Zero Values, Izv. Akad. Nauk USSR, Tekh. Kibern., 1973, no. 2, pp. 152–156.
Zenkov, V.V., Using Weighted Least Squares to Approximate the Discriminant Function with a Cylindrical Surface in Classification Problems, Autom. Remote Control, 2017, vol. 78, no. 9, pp. 9–1662.
Zenkov, V.V., Evaluation of the Posterior Probability of a Class with a Series of Anderson Discriminant Functions, Autom. Remote Control, 2019, vol. 80, no. 3, pp. 3–447.
Zenkov, V.V., Estimating the Probability of a Class at a Point by the Approximation of one Discriminant Function, Autom. Remote Control, 2018, vol. 79, no. 9, pp. 9–1580.
Vorontsov, K.V., Matematicheskie metody obucheniya po pretsedentam (teoriya obucheniya mashin) (Mathematical Methods of Learning by Precedent—Machine Learning Theory), www.machinelearning.ru/wiki/images/6/6d/Voron-ML-1.pdf
Draper, N. and Smit, H., Applied Regression Analysis, New York: Wiley, 1998, 3rd. ed.
Scikit-Learn I.A, Support Vector Machines, https://scikit-learn.org/stable/modules/svm
Sklearn.svm.LinearSVC. https://scikit-learn.Org/0.20/modules/generated/sklearn.svm.LinearSVC.html
UC Irvine Machine Learning Repository, http://mlr.cs.umass.edu/m1/machine-learning-databases/breast-cancer-wisconsin/breast-cancer-Wisconsin.data
UC Irvine Machine Learning Repository, http://mlr.cs.umass.edu/m1/machine-learning-databases/reast-cancer-wisconsin/breast-cancer-Wisconsin.names
Data for the Task at the Traditional Youth School 201A. http://www.machinelearning.ru/wiki/images/e/e1/School-VI-2014-task-3.rar
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper was recommended for publication by V.I. Vasil’ev, a member of the Editorial Board
Russian Text © The Author(s), 2020, published in Avtomatika i Telemekhanika, 2020, No. 1, pp. 159–112.
Rights and permissions
About this article
Cite this article
Zenkov, V.V. Applying an Approximation of the Anderson Discriminant Function and Support Vector Machines for Solving Some Classification Tasks. Autom Remote Control 81, 118–129 (2020). https://doi.org/10.1134/S0005117920010105
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0005117920010105