Sum and Product Kernel Regularization Networks

Abstract

We study the problem of learning from examples (i.e., supervised learning) by means of function approximation theory. Approximation problems formulated as regularized minimization problems with kernel-based stabilizers exhibit easy derivation of solution, in the shape of a linear combination of kernel functions (one-hidden layer feed-forward neural network schemas). Based on Aronszajn’s formulation of sum of kernels and product of kernels, we derive new approximation schemas – Sum Kernel Regularization Network and Product Kernel Regularization Network. We present some concrete applications of the derived schemas, demonstrate their performance on experiments and compare them to classical solutions. For many tasks our schemas outperform the classical solutions.