Efficient induction of numerical constraints

Abstract

In this paper, we address the problem of learning an hyperplane that separates data described by numeric attributes. When data are not linearly separable, the “best” hyperplanes are those that minimize the number of misclassified examples. This topic has already been studied in many works but, as far as we know, the originality of our approach is to be based on Linear Programming technics, which appear to be very efficient. We propose two methods. In the first one, some variables are required to be integers; this enables to get one of the best solutions in terms of the number of misclassified examples, but as a counterpart, solving a Linear Programming problem which includes both real-valued and integer-valued variables is not efficient. In the second method, we relax the constraints for some variables to be integers. In that case, we get only an approximate solution to the problem, but the gain from the point of view of efficiency is so important that the process can be iterated, removing the misclassified examples that are the farthest from the learned hyperplane up to a given threshold. Experiments have been made on randomly generated examples; they show that, in average, the two methods give the same rate of noise.