Machine Learning

, Volume 106, Issue 2, pp 171–195

Fast rates by transferring from auxiliary hypotheses

Article

DOI: 10.1007/s10994-016-5594-4

Cite this article as:
Kuzborskij, I. & Orabona, F. Mach Learn (2017) 106: 171. doi:10.1007/s10994-016-5594-4
  • 230 Downloads

Abstract

In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer. We establish generalization and excess risk bounds, showing that, if the algorithm is fed with a good combination of source hypotheses, generalization happens at the fast rate \(\mathcal {O}(1/m)\) instead of the usual \(\mathcal {O}(1/\sqrt{m})\). On the other hand, if the source hypotheses combination is a misfit for the target task, we recover the usual learning rate. As a byproduct of our study, we also prove a new bound on the Rademacher complexity of the smooth loss class under weaker assumptions compared to previous works.

Keywords

Fast-rate generalization bounds Transfer learning Domain adaptation Rademacher complexity Smooth loss functions Strongly-convex regularizers 

Copyright information

© The Author(s) 2016

Authors and Affiliations

  1. 1.Idiap Research InstituteMartignySwitzerland
  2. 2.Department of Computer ScienceStony Brook UniversityStony BrookUSA

Personalised recommendations