Distributed Learning Automata-based S-learning scheme for classification

  • Morten Goodwin
  • Anis YazidiEmail author
  • Tore Møller Jonassen
Theoretical advances


This paper proposes a novel classifier based on the theory of Learning Automata (LA), reckoned to as PolyLA. The essence of our scheme is to search for a separator in the feature space by imposing an LA-based random walk in a grid system. To each node in the grid, we attach an LA whose actions are the choices of the edges forming a separator. The walk is self-enclosing, and a new random walk is started whenever the walker returns to the starting node forming a closed classification path yielding a many-edged polygon. In our approach, the different LA attached to the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygons, we perform classification by labeling items encircled by a polygon as part of a class using a ray casting function. From a methodological perspective, PolyLA has appealing properties compared to SVM. In fact, unlike PolyLA, the SVM performance is dependent on the right choice of the kernel function (e.g., linear kernel, Gaussian kernel)—which is considered a “black art.” PolyLA, on the other hand, can find arbitrarily complex separator in the feature space. We provide sound theoretical results that prove the optimality of the scheme. Furthermore, experimental results show that our scheme is able to perfectly separate both simple and complex patterns outperforming existing classifiers, such as polynomial and linear SVM, without the need to map the problem to many dimensions or to introduce a “kernel trick.” We believe that the results are impressive, given the simplicity of PolyLA compared to other approaches such as SVM.


Classification Learning Automata Polygons Distributed learning 



  1. 1.
    Caruana R, Niculescu-Mizil A (2006) An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 161–168Google Scholar
  2. 2.
    Caruana R, Karampatziakis N, Yessenalina A (2008) An empirical evaluation of supervised learning in high dimensions. In: Proceedings of the 25th international conference on machine learning. ACM, pp 96–103Google Scholar
  3. 3.
    Madjarov G, Kocev D, Gjorgjevikj D, Džeroski S (2012) An extensive experimental comparison of methods for multi-label learning. Pattern Recognit 45(9):3084–3104CrossRefGoogle Scholar
  4. 4.
    Goodwin M, Tufteland T, Ødesneltvedt G, Yazidi A (2017) Polyaco+: a multi-level polygon-based ant colony optimisation classifier. Swarm Intell 11(3–4):317–346CrossRefGoogle Scholar
  5. 5.
    Agache M, Oommen BJ (2002) Generalized pursuit learning schemes: new families of continuous and discretized learning automata. IEEE Trans Syst Man Cybern Part B Cybern 32(6):738–749CrossRefGoogle Scholar
  6. 6.
    Lakshmivarahan S (1981) Learning algorithms theory and applications. Springer, BerlinzbMATHCrossRefGoogle Scholar
  7. 7.
    Najim K, Poznyak AS (1994) Learning automata: theory and applications. Pergamon Press, OxfordzbMATHGoogle Scholar
  8. 8.
    Narendra KS, Thathachar MAL (1989) Learning automata: an introduction. Prentice-Hall, Inc, Upper Saddle RiverGoogle Scholar
  9. 9.
    Obaidat MS, Papadimitriou GI, Pomportsis AS (2002) Learning automata: theory, paradigms, and applications. IEEE Trans Syst Man Cybern Part B Cybern 32(6):706–709CrossRefGoogle Scholar
  10. 10.
    Poznyak AS, Najim K (1997) Learning automata and stochastic optimization. Springer, BerlinzbMATHGoogle Scholar
  11. 11.
    Thathachar MAL, Sastry PS (2003) Networks of learning automata: techniques for online stochastic optimization. Kluwer Academic, BostonGoogle Scholar
  12. 12.
    Tsetlin ML (1973) Automaton theory and the modeling of biological systems. Academic Press, New YorkzbMATHGoogle Scholar
  13. 13.
    Misra S, Oommen BJ (2004) GPSPA: a new adaptive algorithm for maintaining shortest path routing trees in stochastic networks. Int J Commun Syst 17:963–984CrossRefGoogle Scholar
  14. 14.
    Obaidat MS, Papadimitriou GI, Pomportsis AS, Laskaridis HS (2002) Learning automata-based bus arbitration for shared-edium ATM switches. IEEE Trans Syst Man Cybern Part B 32:815–820CrossRefGoogle Scholar
  15. 15.
    Oommen BJ, Roberts TD (2000) Continuous learning automata solutions to the capacity assignment problem. IEEE Trans Comput 49:608–620CrossRefGoogle Scholar
  16. 16.
    Papadimitriou GI, Pomportsis AS (2000) Learning-automata-based TDMA protocols for broadcast communication systems with bursty traffic. IEEE Commun Lett 4:107–109CrossRefGoogle Scholar
  17. 17.
    Atlassis AF, Loukas NH, Vasilakos AV (2000) The use of learning algorithms in ATM networks call admission control problem: a methodology. Comput Netw 34:341–353CrossRefGoogle Scholar
  18. 18.
    Atlassis AF, Vasilakos AV (2002) The use of reinforcement learning algorithms in traffic control of high speed networks. In: Zimmermann H-J, Tselentis G, van Someren M, Dounias G (eds) Advances in computational intelligence and learning. International Series in Intelligent Technologies, vol 18. Springer, Dordrecht, pp 353–369CrossRefGoogle Scholar
  19. 19.
    Vasilakos AV, Saltouros MP, Atlassis AF, Pedrycz W (2003) Optimizing QoS routing in hierarchical ATM networks using computational intelligence techniques. IEEE Trans Syst Man Cybern Part C 33:297–312CrossRefGoogle Scholar
  20. 20.
    Seredynski F (1998) Distributed scheduling using simple learning machines. Eur J Oper Res 107:401–413zbMATHCrossRefGoogle Scholar
  21. 21.
    Kabudian J, Meybodi MR, Homayounpour MM (2004) Applying continuous action reinforcement learning automata (CARLA) to global training of hidden markov models. In: Proceedings of the international conference on information technology: coding and computing, ITCC’04. Nevada, Las Vegas, pp 638–642Google Scholar
  22. 22.
    Meybodi MR, Beigy H (2002) New learning automata based algorithms for adaptation of backpropagation algorithm pararmeters. Int J Neural Syst 12:45–67CrossRefGoogle Scholar
  23. 23.
    Unsal C, Kachroo P, Bay JS (1997) Simulation study of multiple intelligent vehicle control using stochastic learning automata. Trans Soc Comput Simul 14:193–210Google Scholar
  24. 24.
    Oommen BJ, de St Croix EV (1995) Graph partitioning using learning automata. IEEE Trans Comput 45:195–208MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Collins JJ, Chow CC, Imhoff TT (1995) Aperiodic stochastic resonance in excitable systems. Phys Rev E 52:R3321–R3324CrossRefGoogle Scholar
  26. 26.
    Cook RL (1986) Stochastic sampling in computer graphics. ACM Trans Graph 5:51–72CrossRefGoogle Scholar
  27. 27.
    Barzohar M, Cooper DB (1996) Automatic finding of main roads in aerial images by using geometric-stochastic models and estimation. IEEE Trans Pattern Anal Mach Intell 7:707–722CrossRefGoogle Scholar
  28. 28.
    Brandeau ML, Chiu SS (1989) An overview of representative problems in location research. Manag Sci 35:645–674MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Bettstetter C, Hartenstein H, Prez-Costa X (2004) Stochastic properties of the random waypoint mobility model. J Wirel Netw 10:555–567CrossRefGoogle Scholar
  30. 30.
    Rowlingson BS, Diggle PJ (1991) SPLANCS: spatial point pattern analysis code in S-plus. University of Lancaster, North West Regional Research LaboratoryGoogle Scholar
  31. 31.
    Paola M (1998) Digital simulation of wind field velocity. J Wind Eng Ind Aerodyn 74–76:91–109CrossRefGoogle Scholar
  32. 32.
    Cusumano JP, Kimble BW (1995) A stochastic interrogation method for experimental measurements of global dynamics and basin evolution: application to a two-well oscillator. Nonlinear Dyn 8:213–235CrossRefGoogle Scholar
  33. 33.
    Baddeley A, Turner R (2005) Spatstat: an R package for analyzing spatial point patterns. J Stat Softw 12:1–42CrossRefGoogle Scholar
  34. 34.
    Oommen BJ, Agache M (2001) Continuous and discretized pursuit learning schemes: various algorithms and their comparison. IEEE Trans Syst Man Cybern Part B Cybern 31:277–287CrossRefGoogle Scholar
  35. 35.
    Misra S, Oommen BJ (2005) Dynamic algorithms for the shortest path routing problem: learning automata-based solutions. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1179–1192CrossRefGoogle Scholar
  36. 36.
    Misra S, Oommen BJ (2006) An efficient dynamic algorithm for maintaining all-pairs shortest paths in stochastic networks. IEEE Trans Comput 55(6):686–702CrossRefGoogle Scholar
  37. 37.
    Li H, Mason L, Rabbat M (2009) Distributed adaptive diverse routing for voice-over-ip in service overlay networks. IEEE Trans Netw Serv Manag 6(3):175–189CrossRefGoogle Scholar
  38. 38.
    Mason L (1973) An optimal learning algorithm for s-model environments. IEEE Trans Autom Control 18(5):493–496zbMATHCrossRefGoogle Scholar
  39. 39.
    Beigy H, Meybodi MR (2006) Utilizing distributed learning automata to solve stochastic shortest path problems. Int J Uncertain Fuzziness Knowl Based Syst 14(05):591–615MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Torkestani JA, Meybodi MR (2010) An intelligent backbone formation algorithm for wireless ad hoc networks based on distributed learning automata. Comput Netw 54(5):826–843zbMATHCrossRefGoogle Scholar
  41. 41.
    Torkestani JA, Meybodi MR (2012) Finding minimum weight connected dominating set in stochastic graph based on learning automata. Inf Sci 200:57–77MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Torkestani JA, Meybodi MR (2012) A learning automata-based heuristic algorithm for solving the minimum spanning tree problem in stochastic graphs. J Supercomput 59(2):1035–1054CrossRefGoogle Scholar
  43. 43.
    Thathachar MAL, Sastry PS (2002) Varieties of learning automata: an overview. IEEE Trans Syst Man Cybern Part B Cybern 32(6):711–722CrossRefGoogle Scholar
  44. 44.
    Sastry P, Thathachar M (1999) Learning automata algorithms for pattern classification. Sadhana 24(4):261–292MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Shah S, Sastry PS (1999) New algorithms for learning and pruning oblique decision trees. IEEE Trans Syst Man Cybern Part C (Appl Rev) 29(4):494–505CrossRefGoogle Scholar
  46. 46.
    Thathachar MAL, Sastry PS (1987) Learning optimal discriminant functions through a cooperative game of automata. IEEE Trans Syst Man Cybern 17(1):73–85MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Santharam G, Sastry P, Thathachar M (1994) Continuous action set learning automata for stochastic optimization. J Frankl Inst 331(5):607–628MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Zahiri S (2008) Learning automata based classifier. Pattern Recognit Lett 29(1):40–48CrossRefGoogle Scholar
  49. 49.
    Zeng X, Liu Z (2005) A learning automata based algorithm for optimization of continuous complex functions. Inf Sci 174(3):165–175MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Afshar S, Mosleh M, Kheyrandish M (2013) Presenting a new multiclass classifier based on learning automata. Neurocomputing 104:97–104CrossRefGoogle Scholar
  51. 51.
    Howell M, Gordon T, Brandao F (2002) Genetic learning automata for function optimization. IEEE Trans Syst Man Cyber 32(6):804–815CrossRefGoogle Scholar
  52. 52.
    Barto AG, Anandan P (1985) Pattern-recognizing stochastic learning automata. IEEE Trans Syst Man Cybern 3:360–375MathSciNetzbMATHCrossRefGoogle Scholar
  53. 53.
    Meybodi MR, Beigy H (2002) New learning automata based algorithms for adaptation of backpropagation algorithm parameters. Int J Neural Syst 12(01):45–67zbMATHCrossRefGoogle Scholar
  54. 54.
    Cochran JJ, Cox LA, Keskinocak P, Kharoufeh JP, Smith JC, Stützle T, López‐Ibáñez M, Dorigo M (2011) A concise overview of applications of ant colony optimization. In: Cochran JJ, Cox LA, Keskinocak P, Kharoufeh JP, Smith JC (eds) Wiley encyclopedia of operations research and management science.
  55. 55.
    Dorigo M, Birattari M, Stutzle T (2006) Ant colony optimization. IEEE Comput Intell Mag 1(4):28–39CrossRefGoogle Scholar
  56. 56.
    Goodwin M, Yazidi A (2016) Ant colony optimisation-based classification using two-dimensional polygons. In: International conference on swarm intelligence. Springer, pp 53–64Google Scholar
  57. 57.
    Tufteland T, Ødesneltvedt G, Goodwin M (2016) Optimizing polyaco training with GPU-based parallelization. In: International conference on swarm intelligence. Springer, pp 233–240Google Scholar
  58. 58.
    Goodwin M, Tufteland T, Ødesneltvedt G, Yazidi A (2016) Polyaco+: a many-dimensional polygon-based ant colony optimization classifier for multiple classes. Journal Article (under review)Google Scholar
  59. 59.
    Di Caro G, Dorigo M (1998) Antnet: distributed stigmergetic control for communications networks. J Artif. Intell. Res. 9:317–365zbMATHCrossRefGoogle Scholar
  60. 60.
    Kushner HJ, Clark DS (2012) Stochastic approximation methods for constrained and unconstrained systems, vol 26. Springer, BerlinGoogle Scholar
  61. 61.
    Vázquez-Abad FJ, Mason LG (1996) Adaptive decentralized control under non-uniqueness of the optimal control. Discrete Event Dyn Syst 6(4):323–359zbMATHCrossRefGoogle Scholar
  62. 62.
    Roth SD (1982) Ray casting for modeling solids. Comput Graph Image Process 18(2):109–144CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceOslo Metropolitan UniversityOsloNorway
  2. 2.Department of Information and Communication TechnologyUniversity of AgderKristiansandNorway

Personalised recommendations