Advertisement

Polytope Classifier: A Symbolic Knowledge Extraction from Piecewise-Linear Support Vector Machine

  • Vilen Jumutc
  • Andrey Bondarenko
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6881)

Abstract

This paper describes an extension of a symbolic knowledge extraction approach for Linear Support Vector Machine [1]. The proposed method retrieves a set of concise and interpretable IF-THEN rules from a novel polytope classifier, which can be described as a Piecewise-Linear Support Vector Machine with the successful application for linearly non-separable classification problems. Recent major achievements in rule extraction for kernelized classifiers left some reasonable and unresolved problems in knowledge discovery from nonlinear SVMs. The most comprehensible methods imply constraints that strictly enforce convexity of the searched-through half-space of inducted SVM classifier [2]. Obviously non-convex hyper-surfaces couldn’t be effectively described by a finite set of IF-THEN rules without violating bounds of a constrained non-convex area. In this paper we describe two different approaches for ”learning” a polytope classifier. One of them uses Multi-Surface Method Tree [3] to generate decision half-spaces, while the other one enables clustering-based decomposition of target classes and initiates a separate Linear SVM for every pair of clusters. We claim that the proposed polytope classifier achieves classification rates comparable to a nonlinear SVM and corresponding rule extraction approach helps to extract better rules from linearly non-separable cases in comparison with decision trees and C4.5 rule extraction algorithm.

Keywords

Support Vector Machine Target Class Rule Extraction Linear Support Vector Machine Multiple Kernel Learn 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Fung, G., Sandilya, S., Bharat Rao, R.: Rule Extraction from Linear Support Vector Machines. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, Illinois, USA, April 21-24, pp. 32–40 (2005)Google Scholar
  2. 2.
    Ren, L., Garcez, A.: Symbolic Knowledge Extraction from Support Vector Machines: A Geometric Approach. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008. LNCS, vol. 5507, pp. 335–343. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Mangasarian, O.–L.: Mathematical Programming in Neural Networks. ORSA Journal on Computing 5(4), 349–360 (1993)CrossRefzbMATHGoogle Scholar
  4. 4.
    Nunez, H., Angulo, C., Catala, A.: Rule extraction from support vector machines. In: Proceedings of the 10th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 24-26, pp. 107–112 (2002)Google Scholar
  5. 5.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995)CrossRefzbMATHGoogle Scholar
  6. 6.
    Lloyd, S.–P.: Least squares quantization in PCM. IEEE Transactions on Information Theory 28(2), 129–137 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Grunbaum B.: Convex Polytopes, 2nd edn., prepared by Kaibel, V., Klee, V., Ziegler, G.M. (2003)Google Scholar
  8. 8.
    Bennet K.–P.: Desicion Tree Construction via Linear Programming. In: Proceedings of the 4th Midwest Artificial Intelligence and Cognitive Science Society Conference, pp. 97–101 (1992)Google Scholar
  9. 9.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository, School of Information and Computer Science, Irvine, University of California, USA (2010), http://archive.ics.uci.edu/ml
  10. 10.
    Thompson M.–E.: NDCC: Normally Distributed Clustered Datasets on Cubes, Computer Sciences Department, University of Wisconsin, Madison, USA (2010), http://www.cs.wisc.edu/dmi/svm/ndcc/
  11. 11.
    Rakotomamonjy, A., et al.: SimpleMKL. Journal of Machine Learning Research 9, 2491–2521 (2008)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Vilen Jumutc
    • 1
  • Andrey Bondarenko
    • 1
    • 2
  1. 1.Riga Technical UniversityRigaLatvia
  2. 2.CTCo Ltd.RigaLatvia

Personalised recommendations