A Brief Introduction on Kernels

Abstract

In the previous chapters, we described the Support Vector Machines as a method that creates an optimal hyperplane separating two classes by minimizing the loss via margin maximization. This maximization led to a dual optimization problem resulting in a Lagrangian function which is quadratic and requires simple inequality constraints. The support vectors are responsible for defining the hyperplane, resulting in the support vector classifier f which not only provides a unique solution to the problem, but also ensures learning with tighter guarantees. However this is only possible if the input space is sufficiently linearly separable. On the other hand, many input spaces are, in fact, not linearly separable. In order to overcome this restriction, nonlinear transformations can be used to implicitly obtain a more adequate space.