Forward-backward parallelism in on-line backpropagation
The paper describes the implementation of a systolic array for a multilayer perceptron on ALTERA FLEX10KE FPGAs with a hardware-friendly learning algorithm. A pipelined adaptation of the on-line backpropagation algorithm is shown. It better exploits the parallelism because both the forward and backward phases can be performed simultaneously. As a result, a combined systolic array structure is proposed for both phases. Analytic expressions show that the pipelined version is more efficient than the non-pipelined version. The design is implemented and simulated using VHDL at different levels of abstraction and finally mapped on FPGAs.
Unable to display preview. Download preview PDF.
- S. Shams, and J.L. Gaudiot, “Implementing Regularly Structured Neural Networks on the Dream Machine” IEEE transactions on neural networks, vol. 6, no.2, March 1995, pp. 408–421.Google Scholar
- P. Murtagh, A.C. Tsoi, and N. Bergmann, “Bit-serial array implementation af a multilayer perceptron”, IEEE Proceedings-E, Vol. 140, no. 5, September 1993, pp. 277–288.Google Scholar
- X. Zhang, M. McKenna, J.P. Mesirov, and D Waltz, “An efficient implementation of the backpropagation algorithm on the conection Machine CM-2, Advances in Neural Information Porcessing Systems 2, D.S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 801–809.Google Scholar
- C.R. Rosemberg, and G. Belloch, “An implementation of network learning on the Connection machine”, Connectionist Models an their Implications, D. Waltz and J Feldman, eds., Ablex, Norwood, NJ. 1988.Google Scholar
- R. Gadea, A. Mocholí, “Systolic Implementation of a Pipelined On-Line Backpropagation”, Proc., April 1998.Google Scholar
- D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning internal representations by error backpropagation, Parallel Distributed processing, Vol. 1, MIT Press, Cambridge, MA, 1986, pp. 318–362.Google Scholar
- S.E. Falhman, “Faster learning variations on backpropagation: An empirical study”, Proc. 1988 Connectionist Models Summer School, 1988, pp. 38–50.Google Scholar
- M.R. Zargham, Computer Architecture single and parallel systems, Prentice Hall International Inc. 1996.Google Scholar