Sankhya A

, Volume 79, Issue 2, pp 201–207 | Cite as

Discussion of “concentration for (regularized) empirical risk minimization” by Sara van de Geer and Martin Wainwright

  • Stéphane BoucheronEmail author


Sara van de Geer and Martin Wainwright combine astute convexity arguments and concentration inequalities for suprema of empirical processes to establish generic concentration inequalities for excess penalized risk. This note discusses possible refinements and extensions. In the Gaussian sequence model, concentration of reconstruction error is likely to be improvable and might depend on the effective sparsity of the typical penalized estimator. In the general setting, concentration of excess penalized risk should be complemented by concentration of empirical excess penalized risk. Recent results on penalized least-square estimation pave the way to such a extensions.

Keywords and phrases.

Wilks phenomenon Concentration inequalities Risk estimates Penalization 

AMS (2000) subject classification.

60E15 62G08 62G20 62H30. 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Arlot, S. and Massart, P. (2009). Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res. 10, 245–279.Google Scholar
  2. Bellec, P. C., Lecué, G. and Tsybakov, A. B. (2017). Towards the study of least squares estimators with convex penalty ArXiv e-prints.Google Scholar
  3. Bobkov, S. and Ledoux, M. (1997). Poincaré’s inequalities and Talagrands’s concentration phenomenon for the exponential distribution. Probab. Theory Relat. Fields 107, 383–400.CrossRefzbMATHGoogle Scholar
  4. Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration inequalities Oxford University Press.Google Scholar
  5. Boucheron, S. and Massart, P. (2010). A high-dimensional Wilks phenomenon. Probab. Theory Relat. Fields 148, 1–29.CrossRefzbMATHGoogle Scholar
  6. Bühlmann, P. and van de Geer, S. (2011). Statistics for high-dimensional data. Springer, Heidelberg.CrossRefzbMATHGoogle Scholar
  7. Chatterjee, S. (2014). A new perspective on least squares under convex constraint. Ann. Statist. 42, 6, 2340–2381.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Hastie, T., Tibshirani, R. and Wainwright, M. (2015). Statistical learning with sparsity CRC press.Google Scholar
  9. Koltchinskii, V. (2006). Localized rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics 34, 2593–2656.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Ledoux, M. (2001). The concentration of measure phenomenon. American Mathematical Society, Providence RI.Google Scholar
  11. Lehmann, E. L. and Casella, G. (1998). Theory of point estimation, second edn. Springer Texts in Statistics. Springer-Verlag, New York.zbMATHGoogle Scholar
  12. Massart, P. and Nedelec, E. (2006). Risk bounds for classification. Annals of Statistics 34, 5, 2326.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Saumard, A. (2013). Optimal model selection in heteroscedastic regression using piecewise polynomial functions. Electron. J. Stat. 7, 1184–1223.MathSciNetCrossRefzbMATHGoogle Scholar
  14. van de Geer, S. and Wainwright, M. (2017). On concentration for (regularized) empirical risk minimization Sankhya.Google Scholar

Copyright information

© Indian Statistical Institute 2017

Authors and Affiliations

  1. 1.LPMA UMR 7599 CNRSUniversité Paris-Diderot (Sorbonne Paris Cités)ParisFrance
  2. 2.DMA UMR 8553 CNRSEcole Normale Supérieure (Paris Sciences et Lettres)ParisFrance

Personalised recommendations