Abstract
A systolic array of dedicated processing elements (PEs) is presented as the heart of a multi-model neural-network accelerator. The instruction set of the PEs makes possible to implement several widely-used neural models, including multi-layer Perceptrons with the back-propagation learning rule and Kohonen feature maps. Each PE holds an element of the synaptic weight matrix. An instantaneous swapping mechanism for the weight matrix makes the efficient implementation of neural networks larger than the physical PE array possible. A systolically-flowing instruction accompanies each input vector propagating in the array. This avoids the need of emptying and refilling the array when the operating mode of the array is changed. Fixed point arithmetic is used in the PE. The problem of optimally scaling real variables in fixed-point format is addressed. p ]Both the GENES IV chip, containing a matrix of 2×2 PEs, and an auxiliary arithmetic circuit have been manufactured and successfully tested. The MANTRA I machine has been built around these chips. Peak performances of the full system are between 200 and 400 MCPS in the evaluation phase and between 100 and 200 MCUPS during the learning phase (depending on the algorithm being implemented).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
N.A. Gershenfeld and A.S. Weigend, “The future of time series: Learning and understanding,” A.S. Weigend and N.A. Gershenfeld (Eds.),Time Series Prediction: Forecasting the Future and Understanding the Past, Reading, MA: Addison-Wesley, pp. 1–70, 1993.
J. Hertz, A. Krogh, and R.G. Palmer,Introduction to the Theory of Neural Computation, Santa Fe Institute Studies in Sciences of Complexity, Redwood City, CA: Addison-Wesley, 1991.
C. Lehmann,Réseaux de neurones compétitifs de grandes dimensions pour l'auto-organisation: analyse, synthèse et implantation sur circuits systoliques, Ph.D. Thesis No 1129, Lausanne, École Polytechnique Fédérale de Lausanne, 1993.
F. Blayo,Une implantation systolique des algorithmes connexionnistes, Ph.D. Thesis No 904, Lausanne, École Polytechnique Fédérale de Lausanne, 1990.
P. Thiran, V. Peiris, P. Heim, and B. Hochet, “Quantization effects in digitally behaving circuit implementations of Kohonen networks,”IEEE Transactions on Neural Networks, Volume 5, number 3, pp. 450–458, May 1994.
K. Asanović, and N. Morgan, “Experimental determination of precision requirements for back-propagation training of artificial neural networks,”Proceedings of the 2nd International Conference on Microelectronics for Neural Networks, Munich, pp. 9–15, 1991.
J.L. Holt, and T.E. Baker, “Back propagation simulations using limited precision calculation,”Proceedings of the International Joint Conference on Neural Networks, Seattle, WA, July 1991.
L. Dadda, “Fast multipliers for two's-complement numbers in serial form,”IEEE 7th Symposium on Computer Arithmetic, pp. 57–63, 1985.
P. Ienne and M.A. Viredaz, “Bit-serial multipliers and squarers,”IEEE Transactions on Computers, Volume 43, number 12, pp. 1445–1450, December 1994.
M.A. Viredaz, “MANTRA I: An SIMD processor array for neural computation,” P.P. Spies (Ed.),Proceedings of the Euro-ARCH'93 Conference, München, pp. 99–110, October 1993.
Adaptive Solutions, Inc., Beaverton, Oreg.,CNAPS Server II, 1994, datasheet.
U. Ramacher, J. Beichter, W. Raab, J. Anlauf, N. Brüls, U. Hachmann, and M. Wesseling, “Design of a 1st generation neurocomputer,”VLSI Design of Neural Networks, Norwell, MA: Kluwer Academic Publishers, pp. 271–310, 1991.
M. Yasunaga, N. Masuda, M. Yagyu, M. Asai, K. Shibata, M. Ooyama, M. Yamada, T. Sakaguchi, and M. Hashimoto, “A self-learning neural network composed of 1152 digital neurons in wafer-scale LSIs,”Proceedings of the International Joint Conference on Neural Networks, Seattle, WA, pp. 1844–1849, July 1991.
Y. Sato, K. Shibata, M. Asai, M. Ohki, M. Sugie, T. Sakaguchi, M. Hashimoto, and Y. Kuwabara, “Development of a high-performance general purpose neuro-computer composed of 512 digital neurons,”Proceedings of the International Joint Conference on Neural Networks, volume II, Nagoya, Japan, pp. 1967–1970, October 1993.
U.A. Müller, B. Bäumle, P. Kohler, A. Gunzinger, W. Guggenbühl, “Achieving supercomputer performance for neural net simulation with an array of digital signal processors,”IEEE Micro, pp. 55–65, October 1992.
N. Morgan, J. Beck, P. Kohn, J. Bilmes, E. Allman, and J. Beer, “The Ring Array Processor: A multiprocessing peripheral for connectionist applications,”Journal of Parallel and Distributed Computing, Vol. 14, pp. 248–259, 1992.
N. Mauduit, M. Duranton, J. Gobert, and J.-A. Sirat, “Lneuro 1.0: A piece of hardware LEGO for building neural networks systems,”IEEE Transactions on Neural Networks, Vol. 3, No. 3, pp. 414–422, May 1992.
R. Battiti, “First- and second-order methods for learning: Between steepest descent and newton's method,”Neural Computation, Vol. 4, pp. 141–166, 1992.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ienne, P., Viredaz, M.A. GENES IV: A bit-serial processing element for a multi-model neural-network accelerator. Journal of VLSI Signal Processing 9, 257–273 (1995). https://doi.org/10.1007/BF02407088
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF02407088