Spiking Neural Controllers for Pushing Objects Around

  • Răzvan V. Florian
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4095)


We evolve spiking neural networks that implement a seek-push-release drive for a simple simulated agent interacting with objects. The evolved agents display minimally-cognitive behavior, by switching as a function of context between the three sub-behaviors and by being able to discriminate relative object size. The neural controllers have either static synapses or synapses featuring spike-timing-dependent plasticity (STDP). Both types of networks are able to solve the task with similar efficacy, but networks with plastic synapses evolved faster. In the evolved networks, plasticity plays a minor role during the interaction with the environment and is used mostly to tune synapses when networks start to function.


Neural Network Visual Sensor Contact Sensor Neural Controller Evolutionary Robotic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Florian, R.V.: Autonomous artificial intelligent agents. Technical Report Coneural-03-01, Center for Cognitive and Neural Studies, Cluj, Romania (2003)Google Scholar
  2. 2.
    Maas, W., Bishop, C.M. (eds.): Pulsed neural networks. MIT Press, Cambridge (1999)Google Scholar
  3. 3.
    Gerstner, W., Kistler, W.M.: Spiking neuron models. Cambridge University Press, Cambridge (2002)zbMATHGoogle Scholar
  4. 4.
    Florian, R.V.: Biologically inspired neural networks for the control of embodied agents. Technical Report Coneural-03-03, Center for Cognitive and Neural Studies, Cluj, Romania (2003)Google Scholar
  5. 5.
    DasGupta, B., Schnitger, G.: Analog versus discrete neural networks. Neural Computation 8, 805–818 (1996)CrossRefGoogle Scholar
  6. 6.
    Maass, W., Schnitger, G., Sontag, E.D.: A comparison of the computational power of sigmoid and boolean threshold circuits. In: Roychowdhury, V.P., Siu, K., Orlitsky, A. (eds.) Theoretical Advances in Neural Computation and Learning, pp. 127–151. Kluwer Academic Publishers, Dordrecht (1994)Google Scholar
  7. 7.
    Maas, W.: Networks of spiking neurons: the third generation of neural network models. Neural Networks 10, 1659–1671 (1997)CrossRefGoogle Scholar
  8. 8.
    Floreano, D., Mattiussi, C.: Evolution of spiking neural controllers for autonomous vision-based robots. In: Gomi, T. (ed.) Evolutionary Robotics IV. Springer, Heidelberg (2001)Google Scholar
  9. 9.
    Di Paolo, E.A.: Spike timing dependent plasticity for evolved robots. Adaptive Behavior 10, 243–263 (2002)CrossRefGoogle Scholar
  10. 10.
    Saggie, K., Keinan, A., Ruppin, E.: Solving a Delayed Response Task with Spiking and McCulloch-Pitts Agents. In: Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., Kim, J.T. (eds.) ECAL 2003. LNCS, vol. 2801, pp. 199–208. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    Saggie-Wexler, K., Keinan, A., Ruppin, E.: Neural processing of counting in evolved spiking and mcculloch-pitts agents. Artificial Life 12(1), 1–16 (2005)CrossRefGoogle Scholar
  12. 12.
    Ruppin, E.: Evolutionary embodied agents: A neuroscience perspective. Nature Reviews Neuroscience 3, 132–142 (2002)CrossRefGoogle Scholar
  13. 13.
    Floreano, D., Schoeni, N., Caprari, G., Blynel, J.: Evolutionary bits’n’spikes. In: Standish, R.K., Bedau, M.A., Abbass, H.A. (eds.) Artificial Life VIII: Proceedings of the Eight International Conference on Artificial Life. MIT Press, Boston (2002)Google Scholar
  14. 14.
    Floreano, D., Zufferey, J.C., Mattiussi, C.: Evolving spiking neurons from wheels to wings. In: Proceedings of the 3rd International Symposium on Human and Artificial Intelligence Systems, Fukui, Japan (2002)Google Scholar
  15. 15.
    French, R.L.B., Damper, R.I.: Evolving a nervous system of spiking neurons for a behaving robot. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO 2001), San Francisco, CA, pp. 1099–1106 (2001)Google Scholar
  16. 16.
    French, R.L.B., Damper, R.I.: Evolution of a circuit of spiking neurons for phototaxis in a Braitenberg vehicle. In: Hallam, B., Floreano, D., Hallam, J., Hayes, G., Meyer, J.A. (eds.) From animals to animats 7: Proceedings of the Seventh International Conference on Simulation of Adaptive Behavior, pp. 335–344. MIT Press, Cambridge (2002)Google Scholar
  17. 17.
    Damper, R.I., French, R.L.B.: Evolving Spiking Neuron Controllers for Phototaxis and Phonotaxis. In: Raidl, G.R., Cagnoni, S., Cardalda, J.J.R., Corne, D.W., Gottlieb, J., Guillot, A., Hart, E., Johnson, C.G., Marchiori, E., Meyer, J.-A., Middendorf, M. (eds.) EvoIASP 2003, EvoWorkshops 2003, EvoSTIM 2003, EvoROB/EvoRobot 2003, EvoCOP 2003, EvoBIO 2003, and EvoMUSART 2003. LNCS, vol. 2611, pp. 616–625. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  18. 18.
    Di Paolo, E.A.: Evolving spike-timing dependent plasticity for robot control. In: EPSRC/BBSRC International Workshop: Biologically-inspired Robotics, The Legacy of W. Grey Walter, WGW 2002, Labs, Bristo, August 14 - 16 (2002)Google Scholar
  19. 19.
    Di Paolo, E.A.: Evolving spike-timing dependent plasticity for single-trial learning in robots. Philosophical Transactions of the Royal Society A. 361, 2299–2319 (2003)CrossRefGoogle Scholar
  20. 20.
    Roggen, D., Hofmann, S., Thoma, Y., Floreano, D.: Hardware spiking neural network with run-time reconfigurable connectivity in an autonomous robot. In: 2003 NASA/DoD Conference on Evolvable Hardware (EH 2003), vol. 199 (2003)Google Scholar
  21. 21.
    Van Leeuwen, M., Vreeken, J., Koopman, A.: Evolving vision-based navigation on wheeled robots. Institute for Information and Computing Sciences, Utrecht University (2003)Google Scholar
  22. 22.
    Katada, Y., Ohkura, K., Ueda, K.: Artificial evolution of pulsed neural networks on the motion pattern classification system. In: Proceedings of the 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA), Kobe, Japan, July 16 - 20, 2003, pp. 318–323 (2003)Google Scholar
  23. 23.
    Katada, Y., Ohkura, K., Ueda, K.: An approach to evolutionary robotics using a genetic algorithm with a variable mutation rate strategy. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 952–961. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  24. 24.
    Soula, H., Beslon, G., Favrel, J.: Evolving spiking neural nets to control an animat. In: Proceedings of International Conference of Artificial Neural Networks and Genetic Algorithm 2003, Roanne, France (2003)Google Scholar
  25. 25.
    Hagras, H., Pounds-Cornish, A., Colley, M., Callaghan, V., Clarke, G.: Evolving spiking neural network controllers for autonomous robots. In: Proceedings of the 2004 IEEE International Conference on Robotics and Automation, New Orleans, USA (2004)Google Scholar
  26. 26.
    Federici, D.: A regenerating spiking neural network. Neural Networks 18(5-6), 746–754 (2005)CrossRefGoogle Scholar
  27. 27.
    Federici, D.: Evolving developing spiking neural networks. In: Proceedings of CEC 2005 - IEEE Congress on Evolutionary Computation (2005)Google Scholar
  28. 28.
    Damper, R., Scutt, T.: Biologically-motivated neural learning in situated systems. In: Proceedings of the 1998 IEEE International Symposium on Circuits and Systems (ISCAS 1998) (1998)Google Scholar
  29. 29.
    Damper, R., French, R.L.B., Scutt, T.: Arbib: An autonomous robot based on inspirations from biology. Robotics and Autonomous Systems 31(4), 247–274 (2000)CrossRefGoogle Scholar
  30. 30.
    Soula, H., Alwan, A., Beslon, G.: Obstacle avoidance learning in a spiking neural network. In: Last Minute Results of Simulation of Adaptive Behavior, Los Angeles, CA (2004)Google Scholar
  31. 31.
    Soula, H., Alwan, A., Beslon, G.: Learning at the edge of chaos: Temporal coupling of spiking neuron controller of autonomous robotic. In: Proceedings of AAAI Spring Symposia on Developmental Robotics, Stanford, CA (2005)Google Scholar
  32. 32.
    Florian, R.V.: A reinforcement learning algorithm for spiking neural networks. In: Zaharie, D., Petcu, D., Negru, V., Jebelean, T., Ciobanu, G., Cicortaş, A., Abraham, A., Paprzycki, M. (eds.) Proceedings of the Seventh International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2005), pp. 299–306. IEEE Computer Society, Los Alamitos (2005)Google Scholar
  33. 33.
    Florian, R.V.: Thyrix: A simulator for articulated agents capable of manipulating objects. Technical Report Coneural-03-02, Center for Cognitive and Neural Studies, Cluj, Romania (2003)Google Scholar
  34. 34.
    Pfeifer, R., Scheier, C.: Understanding intelligence. MIT Press, Cambridge (1999)Google Scholar
  35. 35.
    Mureşan, R.C., Ignat, I.: The Neocortex neural simulator: A modern design. In: International Conference on Intelligent Engineering Systems, Cluj-Napoca, Romania, September 19-21 (2004)Google Scholar
  36. 36.
    Markram, H., Lübke, J., Frotscher, M., Sakmann, B.: Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275(5297), 213–215 (1997)CrossRefGoogle Scholar
  37. 37.
    Bi, G.Q., Poo, M.M.: Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience 18(24), 10464–10472 (1998)Google Scholar
  38. 38.
    Bi, G.Q.: Spatiotemporal specificity of synaptic plasticity: cellular rules and mechanisms. Biological Cybernetics 87, 319–332 (2002)zbMATHCrossRefGoogle Scholar
  39. 39.
    Dan, Y., Poo, M.M.: Spike timing-dependent plasticity of neural circuits. Neuron 44, 23–30 (2004)CrossRefGoogle Scholar
  40. 40.
    Song, S., Miller, K.D., Abbott, L.F.: Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3, 919–926 (2000)CrossRefGoogle Scholar
  41. 41.
    Turney, P.: Myths and legends of the Baldwin effect. In: Proceedings of the Workshop on Evolutionary Computing and Machine Learning at the 13th International Conference on Machine Learning (ICML 1996), Bari, Italy, pp. 135–142 (1996)Google Scholar
  42. 42.
    Beer, R.: Toward the evolution of dynamical neural networks for minimally cognitive behavior. In: Maes, P., Mataric, M., Meyer, J., Pollack, J., Wilson, S. (eds.) From animals to animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, pp. 421–429. MIT Press, Cambridge (1996)Google Scholar
  43. 43.
    Slocum, A.C., Downey, D.C., Beer, R.D.: Further experiments in the evolution of minimally cognitive behavior: From perceiving affordances to selective attention. In: Meyer, J.A., Berthoz, A., Floreano, D., Roitblat, H.L., Wilson, S.W. (eds.) From animals to animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior, pp. 430–439. MIT Press, Cambridge (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Răzvan V. Florian
    • 1
    • 2
  1. 1.Center for Cognitive and Neural Studies (Coneural)Cluj-NapocaRomania
  2. 2.Institute for Interdisciplinary Experimental ResearchBabeş-Bolyai UniversityCluj-NapocaRomania

Personalised recommendations