Abstract
We will discuss the role of penalized empirical risk minimization with convex 3 penalties in sparse recovery problems. This includes the `1-norm (LASSO) penalty 4 as well as strictly convex and smooth penalties, such as the negative entropy penalty 5 for sparse recovery in convex hulls. The goal is to show that, when the target 6 function can be well approximated by a “sparse” linear combination of functions 7 from a given dictionary, then solutions of penalized empirical risk minimization 8 problems with `1 and some other convex penalties are “approximately sparse” and 9 they approximate the target function with an error that depends on the “sparsity”. 10 As a result of this analysis, we derive sparsity oracle inequalities showing the 11 dependence of the excess risk of the empirical solution on the underlying sparsity 12 of the problem. These inequalities also involve various distribution dependent 13 geometric characteristics of the dictionary (such as restricted isometry constants 14 and alignment coefficients) and the error of sparse recovery crucially depends on 15 the geometry of the dictionary.
Keywords
- Convex Hull
- Loss Function
- Multiple Kernel Learning
- Empirical Risk Minimization
- Sparse Recovery
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Koltchinskii, V. (2011). Convex Penalization in Sparse Recovery. In: Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Mathematics(), vol 2033. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22147-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-22147-7_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22146-0
Online ISBN: 978-3-642-22147-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)
