Skip to main content

Convex Penalization in Sparse Recovery

  • 1753 Accesses

Part of the Lecture Notes in Mathematics book series (LNMECOLE,volume 2033)

Abstract

We will discuss the role of penalized empirical risk minimization with convex 3 penalties in sparse recovery problems. This includes the `1-norm (LASSO) penalty 4 as well as strictly convex and smooth penalties, such as the negative entropy penalty 5 for sparse recovery in convex hulls. The goal is to show that, when the target 6 function can be well approximated by a “sparse” linear combination of functions 7 from a given dictionary, then solutions of penalized empirical risk minimization 8 problems with `1 and some other convex penalties are “approximately sparse” and 9 they approximate the target function with an error that depends on the “sparsity”. 10 As a result of this analysis, we derive sparsity oracle inequalities showing the 11 dependence of the excess risk of the empirical solution on the underlying sparsity 12 of the problem. These inequalities also involve various distribution dependent 13 geometric characteristics of the dictionary (such as restricted isometry constants 14 and alignment coefficients) and the error of sparse recovery crucially depends on 15 the geometry of the dictionary.

Keywords

  • Convex Hull
  • Loss Function
  • Multiple Kernel Learning
  • Empirical Risk Minimization
  • Sparse Recovery

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   59.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   79.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vladimir Koltchinskii .

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Koltchinskii, V. (2011). Convex Penalization in Sparse Recovery. In: Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Mathematics(), vol 2033. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22147-7_8

Download citation