Pattern Classification via Single Spheres

  • Jigang Wang
  • Predrag Neskovic
  • Leon N. Cooper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3735)

Abstract

Previous sphere-based classification algorithms usually need a number of spheres in order to achieve good classification performance. In this paper, inspired by the support vector machines for classification and the support vector data description method, we present a new method for constructing single spheres that separate data with the maximum separation ratio. In contrast to previous methods that construct spheres in the input space, the new method constructs separating spheres in the feature space induced by the kernel. As a consequence, the new method is able to construct a single sphere in the feature space to separate patterns that would otherwise be inseparable when using a sphere in the input space. In addition, by adjusting the ratio of the radius of the sphere to the separation margin, it can provide a series of solutions ranging from spherical to linear decision boundaries, effectively encompassing both the support vector machines for classification and the support vector data description method. Experimental results show that the new method performs well on both artificial and real-world datasets.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cooper, P.W.: The hypersphere in pattern recognition. Information and Control 5, 324–346 (1962)MATHCrossRefGoogle Scholar
  2. 2.
    Cooper, P.W.: Note on adaptive hypersphere decision boundary. IEEE Transactions on Electronic Computers, 948–949 (1966)Google Scholar
  3. 3.
    Batchelor, B.G., Wilkins, B.R.: Adaptive discriminant functions. Pattern Recognition 42, 168–178 (1968)Google Scholar
  4. 4.
    Batchelor, B.G.: Practical Approach to Pattern Classification. Plenum, New York (1974)Google Scholar
  5. 5.
    Reilly, D.L., Cooper, L.N., Elbaum, C.: A neural model for category learning. Biological Cybernetics 45, 35–41 (1982)CrossRefGoogle Scholar
  6. 6.
    Scofield, C.L., Reilly, D.L., Elbaum, C., Cooper, L.N.: Pattern class degeneracy in an unrestricted storage density memory. In: Anderson, D.Z. (ed.) Neural Information Processing Systems, pp. 674–682. American Institute of Physics, Denver (1987)Google Scholar
  7. 7.
    Marchand, M., Shawe-Taylor, J.: The set covering machine. Journal of Machine Learning Research 3, 723–746 (2002)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Haussler, D. (ed.) Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152 (1992)Google Scholar
  9. 9.
    Cortes, C., Vapnik, V.N.: Support vector networks. Machine Learning 20, 273–297 (1995)MATHGoogle Scholar
  10. 10.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)MATHGoogle Scholar
  11. 11.
    Schölkopf, B., Burges, C., Vapnik, V.: Extracting support data for a given task. In: Proceedings of First International Conference on Knowledge Discovery and Data Mining, pp. 252–257 (1995)Google Scholar
  12. 12.
    Tax, D.M.J., Duin, R.P.W.: Data domain description by support vectors. In: Verleysen, M., ed.: Proceedings ESANN, Brussels, pp. 251–256. D. Facto Press (1999)Google Scholar
  13. 13.
    Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)Google Scholar
  14. 14.
    Vapnik, V.N.: Estimation of Dependence Based on Empirical Data. Springer, Berlin (1982)Google Scholar
  15. 15.
    Osuna, E., Freund, R., Girosi, R.: Support vector machines: training and applications. A.I. Memo AIM - 1602. MIT A.I. Lab (1996)Google Scholar
  16. 16.
    Glineur, F.: Pattern separation via ellipsoids and conic programming, Mémoire de D.E.A., Faculté Polytechnique de Mons, Mons, Belgium (September 1998)Google Scholar
  17. 17.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J.: Estimating the support of a high-dimensional distribution. Neural Computation 13, 1443–1471 (2001)MATHCrossRefGoogle Scholar
  18. 18.
    Blake, C., Merz, C.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Jigang Wang
    • 1
  • Predrag Neskovic
    • 1
  • Leon N. Cooper
    • 1
  1. 1.Department of PhysicsInstitute for Brain and Neural SystemsProvidenceUSA

Personalised recommendations