Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems

École d’Été de Probabilités de Saint-Flour XXXVIII-2008

  • Vladimir Koltchinskii

Part of the Lecture Notes in Mathematics book series (LNM, volume 2033)

Also part of the Ecole d'Eté Probabilit.Saint-Flour book sub series (LNMECOLE, volume 2033)

Table of contents

  1. Front Matter
    Pages i-ix
  2. Vladimir Koltchinskii
    Pages 1-16
  3. Vladimir Koltchinskii
    Pages 17-32
  4. Vladimir Koltchinskii
    Pages 59-79
  5. Vladimir Koltchinskii
    Pages 81-97
  6. Vladimir Koltchinskii
    Pages 121-149
  7. Vladimir Koltchinskii
    Pages 151-189
  8. Vladimir Koltchinskii
    Pages 191-234
  9. Back Matter
    Pages 235-254

About this book

Introduction

The purpose of these lecture notes is to provide an introduction to the general theory of empirical risk minimization with an emphasis on excess risk bounds and oracle inequalities in penalized problems. In recent years, there have been new developments in this area motivated by the study of new classes of methods in machine learning such as large margin classification methods (boosting, kernel machines). The main probabilistic tools involved in the analysis of these problems are concentration and deviation inequalities by Talagrand along with other methods of empirical processes theory (symmetrization inequalities, contraction inequality for Rademacher sums, entropy and generic chaining bounds). Sparse recovery based on l_1-type penalization and low rank matrix recovery based on the nuclear norm penalization are other active areas of research, where the main problems can be stated in the framework of penalized empirical risk minimization, and concentration inequalities and empirical processes tools have proved to be very useful.

Keywords

62J99, 62H12, 60B20, 60G99 concentration inequalities empirical processes low rank matrix recovery sparse recovery

Authors and affiliations

  • Vladimir Koltchinskii
    • 1
  1. 1.School of MathematicsGeorgia Institute of TechnologyAtlantaUSA

Bibliographic information

  • DOI https://doi.org/10.1007/978-3-642-22147-7
  • Copyright Information Springer-Verlag Berlin Heidelberg 2011
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Mathematics and Statistics
  • Print ISBN 978-3-642-22146-0
  • Online ISBN 978-3-642-22147-7
  • Series Print ISSN 0075-8434
  • Series Online ISSN 1617-9692
  • About this book