Discussion of “concentration for (regularized) empirical risk minimization” by Sara van de Geer and Martin Wainwright
- 67 Downloads
Sara van de Geer and Martin Wainwright combine astute convexity arguments and concentration inequalities for suprema of empirical processes to establish generic concentration inequalities for excess penalized risk. This note discusses possible refinements and extensions. In the Gaussian sequence model, concentration of reconstruction error is likely to be improvable and might depend on the effective sparsity of the typical penalized estimator. In the general setting, concentration of excess penalized risk should be complemented by concentration of empirical excess penalized risk. Recent results on penalized least-square estimation pave the way to such a extensions.
Keywords and phrases.Wilks phenomenon Concentration inequalities Risk estimates Penalization
AMS (2000) subject classification.60E15 62G08 62G20 62H30.
Unable to display preview. Download preview PDF.
- Arlot, S. and Massart, P. (2009). Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10, 245–279.Google Scholar
- Bellec, P. C., Lecué, G. and Tsybakov, A. B. (2017). Towards the study of least squares estimators with convex penalty ArXiv e-prints.Google Scholar
- Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration inequalities Oxford University Press.Google Scholar
- Hastie, T., Tibshirani, R. and Wainwright, M. (2015). Statistical learning with sparsity CRC press.Google Scholar
- Ledoux, M. (2001). The concentration of measure phenomenon. American Mathematical Society, Providence RI.Google Scholar
- van de Geer, S. and Wainwright, M. (2017). On concentration for (regularized) empirical risk minimization Sankhya.Google Scholar