Abstract
Sara van de Geer and Martin Wainwright combine astute convexity arguments and concentration inequalities for suprema of empirical processes to establish generic concentration inequalities for excess penalized risk. This note discusses possible refinements and extensions. In the Gaussian sequence model, concentration of reconstruction error is likely to be improvable and might depend on the effective sparsity of the typical penalized estimator. In the general setting, concentration of excess penalized risk should be complemented by concentration of empirical excess penalized risk. Recent results on penalized least-square estimation pave the way to such a extensions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arlot, S. and Massart, P. (2009). Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10, 245–279.
Bellec, P. C., Lecué, G. and Tsybakov, A. B. (2017). Towards the study of least squares estimators with convex penalty ArXiv e-prints.
Bobkov, S. and Ledoux, M. (1997). Poincaré’s inequalities and Talagrands’s concentration phenomenon for the exponential distribution. Probab. Theory Relat. Fields 107, 383–400.
Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration inequalities Oxford University Press.
Boucheron, S. and Massart, P. (2010). A high-dimensional Wilks phenomenon. Probab. Theory Relat. Fields 148, 1–29.
Bühlmann, P. and van de Geer, S. (2011). Statistics for high-dimensional data. Springer, Heidelberg.
Chatterjee, S. (2014). A new perspective on least squares under convex constraint. Ann. Statist. 42, 6, 2340–2381.
Hastie, T., Tibshirani, R. and Wainwright, M. (2015). Statistical learning with sparsity CRC press.
Koltchinskii, V. (2006). Localized rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics 34, 2593–2656.
Ledoux, M. (2001). The concentration of measure phenomenon. American Mathematical Society, Providence RI.
Lehmann, E. L. and Casella, G. (1998). Theory of point estimation, second edn. Springer Texts in Statistics. Springer-Verlag, New York.
Massart, P. and Nedelec, E. (2006). Risk bounds for classification. Annals of Statistics 34, 5, 2326.
Saumard, A. (2013). Optimal model selection in heteroscedastic regression using piecewise polynomial functions. Electron. J. Stat. 7, 1184–1223.
van de Geer, S. and Wainwright, M. (2017). On concentration for (regularized) empirical risk minimization Sankhya.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Boucheron, S. Discussion of “concentration for (regularized) empirical risk minimization” by Sara van de Geer and Martin Wainwright. Sankhya A 79, 201–207 (2017). https://doi.org/10.1007/s13171-017-0113-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13171-017-0113-7