Abstract
We propose a reinforcement learning scheme for random neural networks (RNN) that update network weights based on the relative reward obtained by past actions. This new learning scheme achieves improved performance in random environments under certain conditions, in particular, when the number of actions is small. We also explore hybrid learning structures, where a RNN functions as a meta decision unit that learns how to best delegate the action decision to a second layer of decision units. An extensive simulation study demonstrates the performance of these learning methods applied to minimizing response times in a distributed task assignment scenario. Results are compared to the cognitive packet networks’ reinforcement learning scheme, a reward-only variation, and several non-learning methods. The study reveals that no single method can produce the best performance results in all cases. However, both the generalized reinforcement learning scheme and the application of meta decision units can help to achieve improved performance in specific cases.
Similar content being viewed by others
References
Aiello G, Gaglio S, Re G, Storniolo P, Urso A (2005) The random neural network model for the on-line multicast problem. In: Apolloni B, Marinaro M, Tagliaferri R (eds) Biological and artificial intelligence environments. Springer, Dordrecht, pp 157–164
Atalay V, Gelenbe E (1992) Parallel algorithm for colour texture generation using the random neural network model. Int J Pattern Recognit Artif Intell 06(02n03):437–446. doi:10.1142/S0218001492000266
Atalay V, Gelenbe E, Yalabik N (1992) The random neural network model for texture generation. Int J Pattern Recognit Artif Intell 06(01):131–141. doi:10.1142/S0218001492000072
Bakircioglu H, Gelenbe E, Koccak T (1998) Image processing with the random neural network model. Elektrik 5(1):65–77
Basterrech S, Rubino G (2013) A more powerful random neural network model in supervised learning applications. In: 2013 international conference on soft computing and pattern recognition, SoCPaR 2013, Hanoi, Vietnam, December 15–18, 2013, pp 201–206. doi:10.1109/SOCPAR.2013.7054127
Basterrech S, Mohammed S, Rubino G, Soliman M (2011) Levenbergmarquardt training algorithms for random neural networks. Comput J 54(1):125–135
Bi H, Akinwande O (2015) Multi-path routing in emergency with health-aware classification. In: 2015 2nd international conference on information and communication technologies for disaster management (ICT-DM), pp 109–115. doi:10.1109/ICT-DM.2015.7402026
Bi H, Desmet A, Gelenbe E (2013) Routing emergency evacuees with cognitive packet networks. Springer, Cham, pp 295–303
Busoniu L, hatska RB, Schutter BD (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern C Appl Rev 38(2):156–172. doi:10.1109/TSMCC.2007.913919
Cancela H, Robledo F, Rubino G (2004) A grasp algorithm with rnn based local search for designing a wan access network. Electron Notes Discrete Math 18(Complete):59–65. doi:10.1016/j.endm.2004.06.010
Cramer C, Gelenbe E, Bakircioglu H (1996) Low bit rate video compression with neural networks and temporal sub-samp ling. Proc IEEE 84(10):1529–1543
Cramer C, Gelenbe E, Gelenbe P (1998) Image and video compression. IEEE Potentials 17(1):29–33
Cramer CE, Gelenbe E (2000) Video quality and traffic QoS in learning-based subsampled and receiver-interpolated video sequences. IEEE J Sel Areas Commun 18(2):150–167
Gelenbe E (1989) Random neural networks with positive and negative signals and product form solution. Neural Comput 1(4):502–510
Gelenbe E (1990) Stability of the random neural network model. Neural Comput 2:239–247
Gelenbe E (1992) Une généralisation probabiliste du probleme SAT. Comptes Rendus de l’Académie des Sciences 309 Série II 313:339–3422
Gelenbe E (1993) Learning in the recurrent random neural network. Neural Comput 5(1):154–164. doi:10.1162/neco.1993.5.1.154
Gelenbe E, Batty F (1992) Minimum cost graph covering with the random neural network. Pergamon, New York, pp 139–147
Gelenbe E, Kocak T (2000) Area-based results for mine detection. IEEE Trans Geosci Remote Sens 38(1):12–24
Gelenbe E, Lent R (2004) Power-aware ad hoc cognitive packet networks. Ad Hoc Netw J 2(3):205–216
Gelenbe E, Koubi V, Pekergin F (1993) Dynamical random neural network approach to the traveling salesman problem. In: Proceedings of the IEEE symposium on systems, man and cybernetics, pp 630–635
Gelenbe E, Feng Y, Ranga K, Krishnan R (1996) Neural networks for volumetric MR imaging of the brain. In: Proceedings of international workshop on neural networks for identification, control, robotics and signal/image processing, pp 194–202. doi:10.1109/NICRSP.1996.542760
Gelenbe E, Ghanwani A, Srinivasan V (1997a) Improved neural heuristics for multicast routing. IEEE J Sel Areas Commun 15(2):147–155
Gelenbe E, Ghanwani A, Srinivasan V (1997b) Improved neural heuristics for multicast routing. IEEE J Sel Areas Commun 15(2):147–155. doi:10.1109/49.552065
Gelenbe E, Seref E, Xu Z (1999) Towards networks with intelligent packets. In: Proceedings of the fourteenth international symposium on computer and information sciences, pp 1–11
Gelenbe E, Hussain KF, Abdelbaki H (2000) Random neural network texture model. doi:10.1117/12.382903
Gelenbe E, Lent R, Xu Z (2001a) Design and performance of cognitive packet networks. Perform Eval 46(2,3):155–176
Gelenbe E, Seref E, Xu Z (2001b) Simulation with learning agents. Proc IEEE 89(2):148–157. doi:10.1109/5.910851
Gelenbe E, Lent R, Nunez A (2004) Self-aware networks and QoS. Proc IEEE 9(92):1478–1489
Gelenbe E, Timotheou S, Nicholson D (2010) Random neural network for emergency management. In: Workshop on grand challenges in modeling, simulation and analysis for homeland security (MSAHS’10), Washington, DC
Georgiopoulos M, Li C, Kocak T (2011) Learning in the feed-forward random neural network: a critical review. Perform Eval 68(4):361–384
Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. Trans Neural Netw 5(6):989–993. doi:10.1109/72.329697
Halici U (1997) Reinforcement learning in random neural networks for cascaded decisions. Biosystems 40(1):83–91. doi:10.1016/0303-2647(96)01633-4
Lent R (2007) Linear QoS goals of additive and concave metrics in ad hoc cognitive packet routing. IEEE Trans Syst Man Cybern 6(36):255–1260
Loukas G, Öke G (2007a) A biologically inspired pired denial of service detector using the random neural network. In: IEEE international conference on mobile adhoc and sensor systems, 2007. MASS 2007, pp 1–6
Loukas G, Öke G (2007b) Likelihood ratios and recurrent random neural networks in detection of denial of service attacks. In: Proceedings of international symposium of computer and telecommunication systems (SPECTS ’07), San Diego, CA, USA
Lu R, Shen Y (2005) Image segmentation based on random neural network model and gabor filters. In: 2005 IEEE engineering in medicine and biology 27th annual conference, pp 6464–6467. doi:10.1109/IEMBS.2005.1615979
Martinelli G, Mascioli FMF, Panella M, Rizzi A (2002) Extended random neural networks. In: Proceedings of the 13th Italian workshop on neural nets-revised papers. WIRN VIETRI 2002. Springer, London, pp 75–82. http://dl.acm.org/citation.cfm?id=645961.674072
Mohamed S, Rubino G (2002) A study of real-time packet video quality using random neural networks. IEEE Trans Circuits Syst Video Technol 12(12):1071–1083. doi:10.1109/TCSVT.2002.806808
Narendra KS, Member S, Thathachar MAL (1974) Learning automata–a survey. IEEE Trans Syst Man Cybern 4:323–334
Öke G, Loukas G (2007) A denial of service detector based on maximum likelihood detection and the random neural network. Comput J 50(6):717–727
Öke G, Loukas G, Gelenbe E (2007) Detecting denial of service attacks with bayesian classifiers and the random neural network. In: Fuzzy Systems Conference, 2007. FUZZ-IEEE 2007. IEEE International, pp 1–6
Rubino G, Tirilly P, Varela M (2006) Evaluating users’ satisfaction in packet networks using random neural networks. In: Kollias SD, Stafylopatis A, Duch W, Oja E (eds) Artificial neural networks – ICANN 2006. ICANN 2006. Lecture notes in computer science, vol 4131. Springer, Berlin, Heidelberg
Russell SJ, Norvig P (2003) Artificial intelligence: a modern approach, 2nd edn. Pearson Education, London
Serrano W, Gelenbe E (2016) An intelligent internet search assistant based on the random neural network. Springer, Cham, pp 141–153. doi:10.1007/978-3-319-44944-9_13
Stafylopatis A, Likas A (1992) Pictorial information retrieval using the random neural network. IEEE Trans Softw Eng 18(7):590–600. doi:10.1109/32.148477
Sutton RS, Barto AG (1998) Introduction to reinforcement learning, 1st edn. MIT Press, Cambridge
Zhong Y, Sun D, Wu J (2005) Dynamical random neural network approach to a problem of optimal resource allocation. In: Proceedings of the 8th international conference on artificial neural networks: computational intelligence and bioinspired systems. IWANN’05. Springer, Berlin, pp 1157–1163
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that he has no conflict of interest.
Rights and permissions
About this article
Cite this article
Lent, R. A generalized reinforcement learning scheme for random neural networks. Neural Comput & Applic 31, 2699–2716 (2019). https://doi.org/10.1007/s00521-017-3223-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-017-3223-1