Polytope Classifier: A Symbolic Knowledge Extraction from Piecewise-Linear Support Vector Machine

Abstract

This paper describes an extension of a symbolic knowledge extraction approach for Linear Support Vector Machine [1]. The proposed method retrieves a set of concise and interpretable IF-THEN rules from a novel polytope classifier, which can be described as a Piecewise-Linear Support Vector Machine with the successful application for linearly non-separable classification problems. Recent major achievements in rule extraction for kernelized classifiers left some reasonable and unresolved problems in knowledge discovery from nonlinear SVMs. The most comprehensible methods imply constraints that strictly enforce convexity of the searched-through half-space of inducted SVM classifier [2]. Obviously non-convex hyper-surfaces couldn’t be effectively described by a finite set of IF-THEN rules without violating bounds of a constrained non-convex area. In this paper we describe two different approaches for ”learning” a polytope classifier. One of them uses Multi-Surface Method Tree [3] to generate decision half-spaces, while the other one enables clustering-based decomposition of target classes and initiates a separate Linear SVM for every pair of clusters. We claim that the proposed polytope classifier achieves classification rates comparable to a nonlinear SVM and corresponding rule extraction approach helps to extract better rules from linearly non-separable cases in comparison with decision trees and C4.5 rule extraction algorithm.