A Competitive Learning Approach to Instance Selection for Support Vector Machines

Abstract

Support Vector Machines (SVM) have been applied successfully in a wide variety of fields in the last decade. The SVM problem is formulated as a convex objective function subject to box constraints that needs to be maximized, a quadratic programming (QP) problem. In order to solve the QP problem on larger data sets specialized algorithms and heuristics are required. In this paper we present a new data-squashing method for selecting training instances in support vector learning. Inspired by the growing neural gas algorithm and learning vector quantization we introduce a new, parameter robust neural gas variant to retrieve an initial approximation of the training set containing only those samples that will likely become support vectors in the final classifier. This first approximation is refined in the border areas, defined by neighboring neurons of different classes, yielding the final training set. We evaluate our approach on synthetic as well as real-life datasets, comparing run-time complexity and accuracy to a random sampling approach and the exact solution of the support vector machine. Results show that runtime-complexity can be significantly reduced while achieving the same accuracy as the exact solution and that furthermore our approach does not not rely on data set specific parameterization of the sampling rate like random sampling for doing so. Source code, binary executables as well as the reformatted standard data sets are available for download at http://www.know-center.tugraz.at/forschung/ knowledge_relationship_discovery/downloads_demos/ sngsvm_source_executables