Abstract
Backpropagation is a supervised learning procedure for a class of artificial neural networks which has recently been widely used in training such neural networks to perform relatively nontrivial tasks like text-to-speech conversion or autonomous land vehicle control. However, the slow rate of convergence of the backpropagation algorithm has limited its application to rather small networks and various researchers have implemented parallel versions on a number of different parallel platforms. This work presents experimental speed-up performance results from a parallel implementation of the backpropagation learning algorithm on an Intel iPSC/2 hypercube parallel processor, for such well-known neural nets like NETTalk and extrapolated speed-up results for large scale hypercube systems from analytic performance models of the implementation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cevdet Aykanat, Kemal Oflazer, and Radwan Tahboub. Parallel backpropagation algorithms for medium-to-coarse grain multicomputers. Submitted for Publication, July 1991.
Guy Blelloch and Charles R. Rosenberg. Network learning on the Connection Machine. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, 1987.
K. Diakonikolaou, S. Kollias, D. Kontoravdis, and A. Stafylopatis. Implementation of neural network learning strategies on a transputer-based parallel architecture. Technical report, Compter Science Division, Department of Electrical Engineering, National Technical University of Athens, Greece, 1991.
Deniz Ercoşkun. Parallel implementation of the backpropagation learning algorithm on a hypercube parallel processor. Master’s thesis, Bilkent University — Dept. of Computer Engineering and Information Science, December 1990.
Deniz Ercoşkun and Kemal Oflazer. Experiments with parallel backpropagation on a hypercube parallel processor system. In Proceedings of ICANN-91 — International Conference on Artificial Neural Networks. Helsinki University of Technology, Elsevier Science Publishers, June 1991.
Geoffrey E. Hinton. Connectionist learning procedures. Artificial Intelligence, 1988.
Kevin Knight. A gentle introduction to subsymbolic computation: Connectionism for the AI researcher. Technical Report CMU-CS-89-150, School of Computer Science, Carnegie Mellon University, 1989.
S. Y. Kung and J. N. Hwang. Parallel architectures for artificial neural nets. In Proceedings of IEEE International Conference on Neural Networks, volume 2, pages 165–172, 1988.
Richard P. Lippmann. An introduction to computing with neural nets. IEEE ASSP Magazine, pages 4–22, April 1987.
Dean A. Pomerleau. ALVINN: An autonomous land vehicle in a neural network. In David S. Touretzky, editor, Advances in Neural Information Processing Systems, volume 1. Morgan Kaufman, 1989.
Dean A. Pomerleau, George L. Gusciora, David S. Touretzky, and H. T. Kung. Neural network simulation at Warp speed: How we got 17 million connections per second. In Proceedings of IEEE International Conference on Neural Networks, volume 2, pages 11–143 — 11-150, 1988.
U. Ramacher and J. Beichter. Systolic architectures for fast emulation of artificial neural networks. In Proceedings of International Conference on Systolic Arrays, 1989.
Gareth D. Richards. Implementation of Back-Propagation on a Transputer. Edinburgh Preprint, 1989.
Frank Rosenblatt. Principles of Neurodynamics. Spartan Books, 1962.
David E. Rumelhart, Geoffrey E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In au]David E. Rumelhart and John L. McClelland, editors, Parallel Distributed Processing, volume 1. MIT Press, 1986.
Terence J. Sejnowski and Charles R. Rosenberg. Parallel networks that learn to pronounce English text. Complex Systems, 1:145–168, 1987.
Michael Witbrock and Marco Zagha. An implementation of Back-propagation on GF11, a large SIMD parallel computer. Technical Report CMU-CS-89-208, School of Computer Science, Carnegie Mellon University, December 1989.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1993 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aykanat, C., Oflazer, K., Tahboub, R. (1993). Parallel Implementation of the Backpropagation Algorithm on Hypercube Systems. In: Özgüner, F., Erçal, F. (eds) Parallel Computing on Distributed Memory Multiprocessors. NATO ASI Series, vol 103. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-58066-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-58066-6_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-63460-4
Online ISBN: 978-3-642-58066-6
eBook Packages: Springer Book Archive