Concentration inequalities for the exponential weighting method
The paper is concerned with recovering an unknown vector from noisy data with the help of a family of ordered smoothers . The estimators within this family are aggregated based on the exponential weighting method and the performance of the aggregated estimate is measured by the excess risk controlling deviation of the square losses from the oracle risk. Based on natural statistical properties of ordered smoothers, we propose a novel method for obtaining concentration inequalities for the exponential weighting method.
Keywordsordered smoothers exponential weighting unbiased risk estimation oracle inequality concentration inequality
2010 Mathematics Subject ClassificationPrimary 62C99 secondary 62C10, 62C20, 62J05.
Unable to display preview. Download preview PDF.
- 2.E. Arias-Castro and K. Lounici, “Variable Selection with Exponential Weights and ℓ 0-Penalization,” arXiv:1208.2635.Google Scholar
- 3.H. Akaike, “Information Theory and an Extension of the Maximum Likelihood Principle”, in Proc. 2nd Intern. Symp. Inf. Theory (1973), pp. 267–281.Google Scholar
- 4.O. Catoni, Statistical Learning Theory and Stochastic Optimization, in Lecture Notes in Math. (Springer, Berlin, 2004), Vol. 1851.Google Scholar
- 9.G. Golubev, “Exponential Weighting and Oracle Inequalities for Projection Estimates”, Probl. Inform. Transmission 48(3), 269–280 (2012).Google Scholar
- 15.A. Nemirovski, Topics in Non-Parametric Statistics, in Lectures Notes in Math. (Springer, Berlin, 2000), Vol. 1738.Google Scholar
- 18.C. Stein, “Estimation of the Mean of a Multivariate Normal Distribution”, in Proc. Prague Symp. on Asymptotic Statistics (Prague, Czechoslovakia, 1973), pp. 345–381.Google Scholar