A Perceptron Classifier, Its Correctness Proof and a Probabilistic Interpretation

  • Bernd-Jürgen FalkowskiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10334)


In this paper a fault tolerant probabilistic kernel version with smoothing parameter of Minsky’s perceptron classifier for more than two classes is exhibited and a correctness proof is provided. Moreover it is shown that the resulting classifier approaches optimality. Due to the non-determinism of the algorithm the (approximately) optimal value of a smoothing parameter has to be determined experimentally. The resulting complexity nevertheless allows for an efficient implementation employing for example Java concurrent programming and suitable hardware. In addition a probabilistic interpretation using Bayes Theorem is provided.


Perceptron Classifier for more than 2 classes Bayes decision 



The author is indebted to M. Stern for help with some problems concerning the Java system.


  1. 1.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Block, H.D., Levin, S.A.: On the boundedness of an iterative procedure for solving a system of linear inequalities. In: Proceedings of AMS (1970)Google Scholar
  3. 3.
    Cover, T.M.: Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Electr. Comput. 14, 326–334 (1965)CrossRefzbMATHGoogle Scholar
  4. 4.
    Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and support vector machines. In: Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmanns, D. (eds.) Advances in Large Margin Classifiers. MIT Press, Cambridge (2000)Google Scholar
  5. 5.
    Falkowski, B.-J.: Parallel implementation of certain neural network algorithms. In: Ruan, D., Montero, J., Martinez, L., D’hondt, P., Kerre, E. (eds.) Computational Intelligence in Decision and Control, Proceedings of the 8th International FLINS Conference. World Scientific, Singapore (2008)Google Scholar
  6. 6.
    Falkowski, B.-J.: Minsky revisited, fault tolerant perceptron learning with good generalization properties. In: Miller, L. (ed.) Proceedings of 30th international Conference on Computers and their Applications (CATA 2015). ISCA Publications (2015)Google Scholar
  7. 7.
    Falkowski, B.-J.: A perceptron classifier and corresponding probabilities. In: Ferraro, M.B., Giordani, P., Vantaggi, B., Gagolewski, M., Gil, M.Á., Grzegorzewski, P., Hryniewicz, O. (eds.) Soft Methods for Data Science. AISC, vol. 456, pp. 213–220. Springer, Cham (2017). doi: 10.1007/978-3-319-42972-4_27 CrossRefGoogle Scholar
  8. 8.
    Gallant, S.I.: Perceptron-based learning algorithms. IEEE Trans. Neural Netw. 1(2), 179–191 (1990)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Hands, D.P., Henley, W.E.: Statistical classification methods in consumer credit scoring: a review. J. R. Stat. Soc. Ser. A 160(3), 523–541 (1997)CrossRefGoogle Scholar
  10. 10.
    Kononenko, I.: Machine learning or medical diagnosis: history, state of the art, and perspective. AI Med. 23, 89–109 (2001)Google Scholar
  11. 11.
    Krauth, W., Mezard, M.: Learning algorithms with optimal stability in neural networks, J. Phys. A: Math. Gen. 20 (1987)Google Scholar
  12. 12.
    Meiring, G.A.M., Myburgh, H.C.: A review of intelligent driving style analysis systems and related AI algorithms, senses MDPI (2015)Google Scholar
  13. 13.
    Melvile, D., Webb, G.: Recommender systems. In: Samuel, C., Web, G. (eds.) Encyclopedia of Machine Learning. Springer, USA (2013)Google Scholar
  14. 14.
    Minsky, M.L., Papert, S.: Perceptrons, Expanded edn. MIT Press, Cambridge (1990)zbMATHGoogle Scholar
  15. 15.
    Rojas, R.: Neural Networks. Springer, Berlin (1996)CrossRefzbMATHGoogle Scholar
  16. 16.
    Shashua, A., Levin, A.: Ranking with large margin princple: two approaches. In: NIPS, vol. 14 (2003)Google Scholar
  17. 17.
    Shivani, A., Partha, N.: Generalization bounds for ranking alogorithms via algorithmic stability. J. Mach. Learn. Res. 10, 441–474 (2009)zbMATHMathSciNetGoogle Scholar
  18. 18.
    Schölkopf, B., Herbrich, R., Smola, A.J.: A generalized representer theorem. In: Helmbold, D., Williamson, B. (eds.) COLT 2001. LNCS, vol. 2111, pp. 416–426. Springer, Heidelberg (2001). doi: 10.1007/3-540-44581-1_27 CrossRefGoogle Scholar
  19. 19.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. CUP, New York (2004)CrossRefzbMATHGoogle Scholar
  20. 20.
    Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmanns, D.: Introduction to large margin classifiers. In: Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmanns, D. (eds.) Advances in Large Margin Classifiers. MIT Press, Cambridge (2000)Google Scholar
  21. 21.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Fachhochschule für Ökonomie und Management FOMAugsburgGermany

Personalised recommendations