The Perceptron with Dynamic Margin

  • Constantinos Panagiotakopoulos
  • Petroula Tsampouka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6925)

Abstract

The classical perceptron rule provides a varying upper bound on the maximum margin, namely the length of the current weight vector divided by the total number of updates up to that time. Requiring that the perceptron updates its internal state whenever the normalized margin of a pattern is found not to exceed a certain fraction of this dynamic upper bound we construct a new approximate maximum margin classifier called the perceptron with dynamic margin (PDM). We demonstrate that PDM converges in a finite number of steps and derive an upper bound on them. We also compare experimentally PDM with other perceptron-like algorithms and support vector machines on hard margin tasks involving linear kernels which are equivalent to 2-norm soft margin.

Keywords

Online learning classification maximum margin 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Blum, A.: Lectures on machine learning theory. Carnegie Mellon University, USA, http://www.cs.cmu.edu/~avrim/ML09/lect0126.pdf
  2. 2.
    Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines Cambridge. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  3. 3.
    Duda, R.O., Hart, P.E.: Pattern classsification and scene analysis. Wiley, Chichester (1973)MATHGoogle Scholar
  4. 4.
    Freund, Y., Shapire, R.E.: Large margin classification using the perceptron algorithm. Machine Learning 37(3), 277–296 (1999)CrossRefMATHGoogle Scholar
  5. 5.
    Gentile, C.: A new approximate maximal margin classification algorithm. Journal of Machine Learning Research 2, 213–242 (2001)MathSciNetMATHGoogle Scholar
  6. 6.
    Joachims, T.: Making large-scale SVM learning practical. In: Advances in Kernel Methods-Support Vector Learning. MIT Press, Cambridge (1999)Google Scholar
  7. 7.
    Joachims, T.: Training linear SVMs in linear time. In: KDD, pp. 217–226 (2006)Google Scholar
  8. 8.
    Hsieh, C.-J., Chang, K.-W., Lin, C.-J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear SVM. In: ICML, pp. 408–415 (2008)Google Scholar
  9. 9.
    Ishibashi, K., Hatano, K., Takeda, M.: Online learning of approximate maximum p-norm margin classifiers with bias. In: COLT, pp. 69–80 (2008)Google Scholar
  10. 10.
    Krauth, W., Mézard, M.: Learning algorithms with optimal stability in neural networks. Journal of Physics A20, L745–L752 (1987)Google Scholar
  11. 11.
    Li, Y., Long, P.: The relaxed online maximum margin algorithm. Machine Learning 46(1-3), 361–387 (2002)CrossRefMATHGoogle Scholar
  12. 12.
    Novikoff, A.B.J.: On convergence proofs on perceptrons. In: Proc. Symp. Math. Theory Automata, vol. 12, pp. 615–622 (1962)Google Scholar
  13. 13.
    Panagiotakopoulos, C., Tsampouka, P.: The margin perceptron with unlearning. In: ICML, pp. 855–862 (2010)Google Scholar
  14. 14.
    Panagiotakopoulos, C., Tsampouka, P.: The margitron: A generalized perceptron with margin. IEEE Transactions on Neural Networks 22(3), 395–407 (2011)CrossRefMATHGoogle Scholar
  15. 15.
    Platt, J.C.: Sequential minimal optimization: A fast algorithm for training support vector machines. Microsoft Res. Redmond WA, Tech. Rep. MSR-TR-98-14 (1998)Google Scholar
  16. 16.
    Rosenblatt, F.: The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review 65(6), 386–408 (1958)CrossRefGoogle Scholar
  17. 17.
    Tsampouka, P., Shawe-Taylor, J.: Perceptron-like large margin classifiers. Tech. Rep., ECS, University of Southampton, UK (2005), Obtainable from, http://eprints.ecs.soton.ac.uk/10657
  18. 18.
    Tsampouka, P., Shawe-Taylor, J.: Analysis of generic perceptron-like large margin classifiers. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 750–758. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Tsampouka, P., Shawe-Taylor, J.: Constant rate approximate maximum margin algorithms. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 437–448. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  20. 20.
    Tsampouka, P., Shawe-Taylor, J.: Approximate maximum margin algorithms with rules controlled by the number of mistakes. In: ICML, pp. 903–910 (2007)Google Scholar
  21. 21.
    Vapnik, V.: Statistical learning theory. Wiley, Chichester (1998)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Constantinos Panagiotakopoulos
    • 1
  • Petroula Tsampouka
    • 1
  1. 1.Physics Division, School of TechnologyAristotle University of ThessalonikiGreece

Personalised recommendations