The “moving targets” training algorithm

  • Richard Rohwer
Part II Theory, Algorithms
Part of the Lecture Notes in Computer Science book series (LNCS, volume 412)


A simple method for training the dynamical behavior of a neural network is derived. It is applicable to any training problem in discrete-time networks with arbitrary feedback. The algorithm resembles back-propagation in that an error function is minimized using a gradient-based method, but the optimization is carried out in the hidden part of state space either instead of, or in addition to weight space. A straightforward adaptation of this method to feedforward networks offers an alternative to training by conventional back-propagation. Computational results are presented for some simple dynamical training problems, one of which requires response to a signal 100 time steps in the past.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    K. Birmiwal, P. Sarwal, and S. Sinha, “A new Gradient-Free Learning Algorithm”, Tech. report, Dept. of EE, Southern Illinois U., Carbondale, (1989).Google Scholar
  2. [2]
    T. Grossman, R. Meir, and E. Domany, “Learning by Choice of Internal Representations”, Dept. of Electronics, Weizmann Institute of Science, Rehovot, Israel, (1988).Google Scholar
  3. [3]
    G. L. Heileman, M. Georgiopoulos, and A. K. Brown, “The Minimal Disturbance Back Propagation Algorithm”, Tech. report, Dept. of EE, U. of Central Florida, Orland, (1989).Google Scholar
  4. [4]
    G. Kuhn, “Connected Recognition with a Recurrent Network”, to appear in proc. NEUROSPEECH, 18 May 1989, as special issue of Speech Communication, v. 9, no. 2, (1990).Google Scholar
  5. [5]
    Pearlmutter, B., “Learning State Space Trajectories in Recurrent Neural Networks”, Proc. IEEE IJCNN 89, Washington D. C., Publ. IEEE TAB Neural Network Committee., p. II-365, (1989).Google Scholar
  6. [6]
    R. Fletcher, Practical Methods of Optimization, v1, Wiley, (1980).Google Scholar
  7. [7]
    W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes, The Art of Scientific Computing, Cambridge, (1986).Google Scholar
  8. [8]
    R. Rohwer and B. Forrest, “Training Time Dependence in Neural Networks” Proc. IEEE ICNN, San Diego, p. II-701, (1987).Google Scholar
  9. [9]
    R. Rohwer and S. Renals, “Training Recurrent Networks”, in Neural Networks from Models to Applications, L. Personnaz and G. Dreyfus, eds., I.D.S.E.T., Paris, p. 207, (1989).Google Scholar
  10. [10]
    Rumelhart, D., Hinton, G. and Williams, R., “Learning Internal Representations by Error Propagation” in Parallel Distributed Processing, v. 1, MIT, (1986).Google Scholar
  11. [11]
    R. L. Watrous, “Phoneme Discrimination using Connectionist Networks”, Tech. Report, Dept. of Computer Science, U. Penn., submitted to J. Acoustical Soc. of America, (1989).Google Scholar
  12. [12]
    P. Werbos, Energy Models and Studies, B. Lev, Ed., North Holland, (1983).Google Scholar
  13. [13]
    R. Williams and D. Zipser, “A Learning Algorithm for Continually Running Fully Recurrent Neural Networks”, ICS Report 8805, UC San Diego, (1988).Google Scholar
  14. [14]
    R. Rohwer, “The ‘Moving Targets’ Training Algorithm”, to appear in Proc. DANIP, GMD Bonn, J. Kinderman and A. Linden, Eds., 24–25 April, 1989.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1990

Authors and Affiliations

  • Richard Rohwer
    • 1
  1. 1.Centre for Speech Technology ResearchUniversity of EdinburghEdinburghScotland

Personalised recommendations