Improved Multi-class Support Vector Machines Using Novel Methods of Model Selection and Feature Extraction

Abstract

In this paper, to improve the generalization capability of multi-class SVMs, we propose (1) a novel model selection and (2) feature extraction by SVMs. In (1), unlike the conventional model selection in multi-class SVMs, we determine hyper-parameters, which are kernel parameter and margin parameter, for each separating hyper-plane, separately. Namely, for each separating hyper-plane, we estimate the generalization capability and select optimal values of the hyper-parameters, separately. In (2), we define the weighted vectors of decision functions determined by training multi-class SVMs as the basis vector of the subspace, and we determine the separating hyper-planes in the subspace. Thus, we can determine the new separating hyper-planes during considering the all separating hyper-planes. Using multi-class benchmark data sets, we evaluate the effectiveness of the proposed methods over the conventional method.