Neural Computing and Applications

, Volume 18, Issue 4, pp 369–375 | Cite as

A small spiking neural network with LQR control applied to the acrobot

Original Article

Abstract

This paper presents the results of a computer simulation which, combined a small network of spiking neurons with linear quadratic regulator (LQR) control to solve the acrobot swing-up and balance task. To our knowledge, this task has not been previously solved with spiking neural networks. Input to the network was drawn from the state of the acrobot, and output was torque, either directly applied to the actuated joint, or via the switching of an LQR controller designed for balance. The neural network’s weights were tuned using a (μ + λ)-evolution strategy without recombination, and neurons’ parameters, were chosen to roughly approximate biological neurons.

Keywords

Spiking neural networks Acrobot LQR Evolution 

References

  1. 1.
    Anderson, Moore (1971) Linear optimal control. Prentice Hall, Englewood CliffsGoogle Scholar
  2. 2.
    Beyer H-S (2001) The theory of evolution strategies. Springer, HeidelbergGoogle Scholar
  3. 3.
    Boone G (1997) Minimum-time control of the acrobot. In: Proceedings of IEEE international conference on robotics and automation, vol 4, pp 3281–3287Google Scholar
  4. 4.
    Coulom R (2004) High-accuracy value-function approximation with neural networks applied to the acrobot. European Symposium on Artificial Neural NetworksGoogle Scholar
  5. 5.
    Federici D (2005) Evolving developing spiking neural networks. In: The IEEE congress on evolutionary computation, vol 1, pp 543–550Google Scholar
  6. 6.
    Dario Floreano, Yann Epars, Jean-Christophe Zufferey, Claudio Mattiussi (2006) Evolution of spiking neural circuits in autonomous mobile robots: Research articles. Int J Intell Syst 21(9):1005–1024Google Scholar
  7. 7.
    French RLB, Damper RI (2002) Evolution of a circuit of spiking neurons for phototaxis in a Braitenberg vehicle. In: ICSAB: proceedings of the seventh international conference on simulation of adaptive behavior on From animals to animats. MIT Press, Cambridge, pp 335–344Google Scholar
  8. 8.
    Gerstner W (2001) Pulsed neural networks, Chap. 1: Spiking neurons, pp 3–53. In: Maass and Bishop [15]Google Scholar
  9. 9.
    Gerstner W, Werner M.K (2002) Spiking neuron models: single neurons, populations, plasticity. Formal spiking neuron models, Chap. 4. Cambridge University Press, CambridgeGoogle Scholar
  10. 10.
    Joshi P, Maass W (2005) Movement generation with circuits of spiking neurons. Neural Comput 17(8):1715–1738MATHCrossRefGoogle Scholar
  11. 11.
    Kandel ER, Schwartz JH, Jessell TM (2000) Principles of neural science, 4th edn. McGraw-Hill, New YorkGoogle Scholar
  12. 12.
    Kawada K, Fujisawa S, Obika M, Yamamoto T (2005) Creating swing-up patterns of an acrobot using evolutionary computation. Proceedings of IEEE international symposium on computational intelligence in robotics and automation, CIRA 2005, pp 261–266Google Scholar
  13. 13.
    Lai X, She JH, Ohyama Y, Cai Z (1999) Fuzzy control strategy for acrobots combining model-free andmodel-based control. IEE Proc Control Theory Appl 146(6):505–510CrossRefGoogle Scholar
  14. 14.
    Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRefGoogle Scholar
  15. 15.
    Maass W, Bishop CM (eds) (2001) Pulsed neural networks. MIT Press, CambridgeGoogle Scholar
  16. 16.
    Markram H (2006) The blue brain project. Nat Rev Neurosci 7(2):153–160CrossRefMathSciNetGoogle Scholar
  17. 17.
    Nam TK, Fukuhara Y, Mita T, Yamakita M (2002) Swing-up control and avoiding singular problem of an acrobot system. In: Proceedings of the 41st SICE annual conference, SICE 2002Google Scholar
  18. 18.
    Press WH, Teukolsky ST, Vetterling WT, Flannery BP (2002) Numerical recipes in C: the art of scientific computing, Chap. 16.1, 2nd edn. Cambridge University Press, Cambridge, pp 710–714Google Scholar
  19. 19.
    Spong MW (1995) The swing up control problem for the acrobot. IEEE Control Syst Magaz 15(1):49–55CrossRefGoogle Scholar
  20. 20.
    Xu X, He H (2002) Residual-gradient-based neural reinforcement learning for the optimal control of an acrobot. In: Proceedings of the 2002 IEEE international symposium on intelligent control, pp 758–763Google Scholar
  21. 21.
    Yoshimoto J, Nishimura M, Tokita Y, Ishii S (2005) Acrobot control by learning the switching of multiple controllers. Artif Life Robot 9(2):67–71CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2008

Authors and Affiliations

  • Lukasz Wiklendt
    • 1
  • Stephan Chalup
    • 1
  • Rick Middleton
    • 1
  1. 1.School of Electrical Engineering and Computer ScienceThe University of NewcastleCallaghanAustralia

Personalised recommendations