An Efficient Hardware Implementation of Feed-Forward Neural Networks*
This paper proposes a new way of digital hardware imple- mentation of nonlinear activation functions in feed-forward neural net- works. The basic idea of this new realization is that the nonlinear functions can be implemented using a matrix-vector multiplication. Recently a new approach was proposed for the realization of matrix-vector mul- tiplications which approach can also be applied for implementing the nonlinear functions if the nonlinear functions are approximated by sim- ple basis functions. The paper proposes to use B-spline basis functions to the approximate nonlinear sigmoidal functions, it shows that this ap proximation fulfills the general requirements on the activation functions, presents the details of the proposed hardware implementation, and gives a summary of an extensive study about the effects of B-spline nonlin- ear function realization on the size and the trainability of feed-forward neural networks.
KeywordsMean Square Error Activation Function Hide Neuron Order Spline Nonlinear Activation Function
Unable to display preview. Download preview PDF.
- 1.M. H. Hassoun, Fundamentals of artificial neural networks, MIT Press, 1995.Google Scholar
- 2.T. G. Clarkson and Y. Ding, RAM-Based Neural Networks, chapter Extracting directional information for the recognition of fingerprints by pRAM networks, pp. 174–185, World Scientific, 1998.Google Scholar
- 3.“White blood cell classification & counting with the ZISC, ” http://www.fr.ibm.com/france/cdlab/zblcell.htm, 2000.
- 4.Tamás Szabó, Béla Fehér, and Gäbor Horväth, “Neural network implementation using distributed arithmetic, ” in Proceedings of the International Conference on Knowledge-based Electronic Systems, Adelaide, Australia, 1998, vol. 3, pp. 511–520.Google Scholar
- 5.Tamäs Szabó, Lörinc Antoni, Gäbor Horváth, and Béla Fehér, “An efficient implementation for a matrix-vector multiplier structure, ” in Proceedings of IEEE International Joint Conference on Neural Networtks, IJCNN2000, 2000, vol. II, pp. 49–54.Google Scholar
- 6.Manferd Glesner and Werner Pöchmüller, Neurocomputers, an overview of neural networks in VLSI, Neural Computing. Chapman & Hall, 1994.Google Scholar
- 8.Věra Kurková, “Approximation of functions by neural networks, ” in Proceedings of NC’98, 1998, pp. 29–36.Google Scholar