Machine Learning

, Volume 48, Issue 1, pp 165–187

Kernel Matching Pursuit

  • Pascal Vincent
  • Yoshua Bengio

DOI: 10.1023/A:1013955821559

Cite this article as:
Vincent, P. & Bengio, Y. Machine Learning (2002) 48: 165. doi:10.1023/A:1013955821559


Matching Pursuit algorithms learn a function that is a weighted sum of basis functions, by sequentially appending functions to an initially empty basis, to approximate a target function in the least-squares sense. We show how matching pursuit can be extended to use non-squared error loss functions, and how it can be used to build kernel-based solutions to machine learning problems, while keeping control of the sparsity of the solution. We present a version of the algorithm that makes an optimal choice of both the next basis and the weights of all the previously chosen bases. Finally, links to boosting algorithms and RBF training procedures, as well as an extensive experimental comparison with SVMs for classification are given, showing comparable results with typically much sparser models.

kernel methodsmatching pursuitsparse approximationsupport vector machinesradial basis functionsboosting
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Pascal Vincent
    • 1
  • Yoshua Bengio
    • 1
  1. 1.Department of IROUniversité de MontréalMontrealCanada