Forward-backward parallelism in on-line backpropagation

  • Rafael Gadea Gironés
  • Antonio Mocholí Salcedo
Artificial Neural Nets Simulation and Implementation
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1607)


The paper describes the implementation of a systolic array for a multilayer perceptron on ALTERA FLEX10KE FPGAs with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algorithm is shown. It better exploits the parallelism because both the forward and backward phases can be performed simultaneously. As a result, a combined systolic array structure is proposed for both phases. Analytic expressions show that the pipelined version is more efficient than the non-pipelined version. The design is implemented and simulated using VHDL at different levels of abstraction and finally mapped on FPGAs.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Singer, “Implementations of artificial neural networks on the connection machine”, Parallel Computing, vol. 14, 1990, pp. 305–315.CrossRefGoogle Scholar
  2. [2]
    S. Shams, and J.L. Gaudiot, “Implementing Regularly Structured Neural Networks on the Dream Machine” IEEE transactions on neural networks, vol. 6, no.2, March 1995, pp. 408–421.Google Scholar
  3. [3]
    W-M Iin, V. K. Prasanna, and K. W. Przytula, “Algorithmic Mapping of Neural Network Models onto Parallel SIMD Machines”, IEEE Transactions on Computers, Vol. 40, no. 12, December 1991, pp. 1390–1401.CrossRefGoogle Scholar
  4. [4]
    S.R. Jones, K.M. Sammut, and J. Hunter, “Learning in Linear Systolic Neural Network Engines: Analysis and Implementation”, Transactions on Neural Netwoks, Vol. 5, no. 4, July 1994, pp. 584–593.CrossRefGoogle Scholar
  5. [5]
    D. Naylor, S. Jones, and D. Myers, “Backpropagation in Linear Arrays—A performance Analysis and Optimization”, IEEE Transactions on Neural Networks, Vol. 6, no. 3, May 1995, pp. 583–595.CrossRefGoogle Scholar
  6. [6]
    P. Murtagh, A.C. Tsoi, and N. Bergmann, “Bit-serial array implementation af a multilayer perceptron”, IEEE Proceedings-E, Vol. 140, no. 5, September 1993, pp. 277–288.Google Scholar
  7. [7]
    X. Zhang, M. McKenna, J.P. Mesirov, and D Waltz, “An efficient implementation of the backpropagation algorithm on the conection Machine CM-2, Advances in Neural Information Porcessing Systems 2, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 801–809.Google Scholar
  8. [8]
    C.R. Rosemberg, and G. Belloch, “An implementation of network learning on the Connection machine”, Connectionist Models an their Implications, D. Waltz and J Feldman, eds., Ablex, Norwood, NJ. 1988.Google Scholar
  9. [9]
    A. Petrowski, G. Dreyfus, and C. Girault, “Performance Analysis of a Pipelined Backpropagation Parallel Algorithm”, IEEE Transaction on Neural Networks, Vol.4, no. 6, November 1993, pp. 970–981.CrossRefGoogle Scholar
  10. [10]
    R. Gadea, A. Mocholí, “Systolic Implementation of a Pipelined On-Line Backpropagation”, Proc., April 1998.Google Scholar
  11. [11]
    D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning internal representations by error backpropagation, Parallel Distributed processing, Vol. 1, MIT Press, Cambridge, MA, 1986, pp. 318–362.Google Scholar
  12. [12]
    S.E. Falhman, “Faster learning variations on backpropagation: An empirical study”, Proc. 1988 Connectionist Models Summer School, 1988, pp. 38–50.Google Scholar
  13. [13]
    M.R. Zargham, Computer Architecture single and parallel systems, Prentice Hall International Inc. 1996.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Rafael Gadea Gironés
  • Antonio Mocholí Salcedo

There are no affiliations available

Personalised recommendations