Abstract
Existing mathematical model of learning requires using training data find in a given subset of admissible function the function that minimizes the expected loss. In the paper this setting is called Second selection problem. Mathematical model of learning in this paper along with Second selection problem requires to solve the so-called First selection problem where using training data one first selects from wide set of function in Hilbert space an admissible subset of functions that include the desired function and second selects in this admissible subset a good approximation to the desired function. Existence of two selection problems reflects fundamental property of Hilbert space, existence of two different concepts of convergence of functions: weak convergence (that leads to solution of the First selection problem) and strong convergence (that leads to solution of the Second selection problem). In the paper we describe simultaneous solution of both selection problems for functions that belong to Reproducing Kernel Hilbert space. The solution is obtained in closed form.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Vapnik, V.N. and Chervonenkis, A.J., Necessary and Sufficient Convergence of Relative Frequencies of events to Their Probabilities, Theor. Prob. App., 1971, vol. 16, no. 2, pp. 264–280.
Vapnik, V.N. and Chervonenkis, A.J., Teoriya raspoznavaniya obrazov (Theory of Pattern Recognition), Moscow: Nauka, 1974.
Vapnik, V., The Nature of Statistical Learning Theory, New York: Springer, 1995.
Vapnik, V.N., Statistical Learning Theory, New York: Wiley, 1998.
Vapnik, V.N., Estimation of Dependencies Based on Empirical Data, Moscow: Nauka, 1979.
Devroy, L., Geofry, L., and Lugosi, G., A Probabilistic Theory of Pattern Recognition, New York: Springer, 1996.
Tikhonov, A.N. and Arsenin, V.Ya., Solutions of Ill-Posed Problems, Washington: Winston, 1977.
Vapnik, V. and Izmailov, R., Rethinking Statistical Learning Theory: Learning Using Statistical Invariants, Machine Learning, 2018, vol. 108, pp. 381–423.
Author information
Authors and Affiliations
Corresponding author
Additional information
In the memory of outstanding scientist and remarkable person Ja.Z. Tsypkin
Russian Text © The Author(s), 2019, published in Avtomatika i Telemekhanika, 2019, No. 11, pp. 24–58.
Rights and permissions
About this article
Cite this article
Vapnik, V.N. Complete Statistical Theory of Learning. Autom Remote Control 80, 1949–1975 (2019). https://doi.org/10.1134/S000511791911002X
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S000511791911002X