Abstract
For a finite function class, we describe the large sample limit of the sequential Rademacher complexity in terms of the viscosity solution of a G-heat equation. In the language of Peng’s sublinear expectation theory, the same quantity equals to the expected value of the largest order statistics of a multidimensional G-normal random variable. We illustrate this result by deriving upper and lower bounds for the asymptotic sequential Rademacher complexity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S. Boucheron, G. Lugosi, and P. Massart, Concentration inequalities: a nonasymptotic theory of independence, Oxford University Press, Oxford, 2013.
N. Cesa-Bianchi and G. Lugosi, Prediction, learning, and games, Cambridge University Press, Cambridge, 2006.
M.G. Crandall, H. Ishii, and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. 27 (1992), 1–67.
F. Da Lio and O. Ley, Uniqueness results for second-order Bellman-Isaacs equations under quadratic growth conditions and applications, SIAM J. Control Optim. 45 (2006), 74–106.
Y. Giga, Surface Evolution Equations. A Level Set Approach, Birkhäuser, Basel, 2006.
M.B. Marcus and J. Rosen, Markov processes, Gaussian processes, and local times, Cambridge University Press, Cambridge, 2006.
S. Peng, A new central limit theorem under sublinear expectations, Preprint. arXiv:0803.2656 [math.PR], 2008.
S. Peng, Multi-dimensional \(G\)-Brownian motion and related stochastic calculus under \(G\)-expectation, Stoch. Proc. Appl. 118 (2008), 2223–2253.
S. Peng, Nonlinear expectations and stochastic calculus under uncertainty, Preprint. arXiv:1002.4546 [math.PR], 2010.
A. Rakhlin, K. Sridharan, and A. Tewari, Online learning: random averages, combinatorial parameters, and learnability, Adv. Neural Inf. Process. Syst. 23 (2010), 1984–1992.
A. Rakhlin, K. Sridharan, and A. Tewari, Online learning via sequential complexities, J. Mach. Learn. Res. 16 (2015), 155–186.
A. Rakhlin, K. Sridharan, and A. Tewari, Sequential complexities and uniform martingale laws of large numbers, Probab. Theory Relat. Fields 161 (2015), 111–153.
D.B. Rokhlin, Central limit theorem under uncertain linear transformations, Stat. Probabil. Lett. 107 (2015), 191–198.
C. Sammut and G.I. Webb (eds.), Encyclopedia of Machine Learning, Springer, New York, 2011.
M. Steele, Probability Theory and Combinatorial Optimization, SIAM, Philadelphia, 1997.
T. Strömberg, Exponentially growing solutions of parabolic Isaacs’ equations, J. Math. Anal. Appl. 348 (2008), 337–345.
M. Talagrand, The Generic Chaining. Upper and Lower Bounds for Stochastic Processes, Springer, Berlin, 2005.
V.V. V’yugin, VS Dimension, Fat-Shattering Dimension, Rademacher Averages, and Their Applications, In: V. Vovk, H. Papadopoulos, and A. Gammerman (eds.), Measures of Complexity: Festschrift in Honor of A. Chervonenkis, Chapter 6, 57–74, Springer, Cham, 2015.
J. Yong and X.Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer, New York, 1999.
Acknowledgements
The research is supported by Southern Federal University, project 213.01-07-2014/07.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rokhlin, D.B. Asymptotic sequential Rademacher complexity of a finite function class. Arch. Math. 108, 325–335 (2017). https://doi.org/10.1007/s00013-016-1002-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00013-016-1002-3