State Prediction: A Constructive Method to Program Recurrent Neural Networks

  • René Felix Reinhart
  • Jochen Jakob Steil
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6791)

Abstract

We introduce a novel technique to program desired state sequences into recurrent neural networks in one shot. The basic methodology and its scalability to large and input-driven networks is demonstrated by shaping attractor landscapes, transient dynamics and programming limit cycles. The approach unifies programming of transient and attractor dynamics in a generic framework.

Keywords

recurrent neural networks input-driven dynamics learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Atiya, A.F., Parlos, A.G.: New Results on Recurrent Network Training. IEEE Trans. Neural Networks 11, 697–709 (2000)CrossRefGoogle Scholar
  2. 2.
    Hopfield, J.J.: Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Reinhart, R.F., Steil, J.J.: A Constrained Regularization Approach for Input-Driven Recurrent Neural Networks. Differ. Equ. Dyn. Syst., 1–20 (2010)Google Scholar
  4. 4.
    Reinhart, R.F., Steil, J.J.: Reservoir Regularization Stabilizes Learning of Echo State Networks with Output Feedback. In: European Symp. on ANNs (2011)Google Scholar
  5. 5.
    Jaeger, H.: Reservoir Self-Control for Achieving Invariance Against Slow Input Distortions. Technical report, Jacobs University (2010)Google Scholar
  6. 6.
    Lecun, Y., Cortes, C., http://yann.lecun.com/exdb/mnist/
  7. 7.
    Beer, R.D.: Dynamical Approaches to Cognitive Science. Trends in Cognitive Sciences 4, 91–99 (2000)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Beer, R.D.: On the Dynamics of Small Continuous-Time Recurrent Neural Networks. Adaptive Behavior 3, 469–509 (1995)CrossRefGoogle Scholar
  9. 9.
    Haschke, R., Steil, J.J.: Input Space Bifurcation Manifolds of Recurrent Neural Networks. Neurocomputing 64, 25–38 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • René Felix Reinhart
    • 1
  • Jochen Jakob Steil
    • 1
  1. 1.Research Institute for Cognition and Robotics (CoR-Lab) & Faculty of TechnologyBielefeld UniversityBielefeldGermany

Personalised recommendations