A scalable performance prediction method for parallel neural network simulations
A performance prediction method is presented for indicating the performance range of MIMD parallel processor systems for neural network simulations. The total execution time of a parallel application is modeled as the sum of its calculation and communication times. The method is scalable because based on the times measured on one processor and one communication link, the performance, speedup, and efficiency can be predicted for a larger processor system. It is validated quantitatively by applying it to two popular neural networks, backpropagation and the Kohonen self-organizing feature map, decomposed on a GCel-512, a 512 transputer system. Agreement of the model with the measurements is within 9%.
Unable to display preview. Download preview PDF.
- 1.L-C. Chu and B.W. Wah. Optimal Mapping of Neural Network Learning on Message-Passing Multicomputers. Journal of Parallel and Distributed Computing, 14:319–339, 1992.Google Scholar
- 2.A.J.G. Hey. The Genesis Distributed Memory Benchmarks. Parallel Computing, 17(2):1275–1283, 1991.Google Scholar
- 3.T. Kohonen. Self-Organization and Associative Memory. Springer Verlag, Berlin, second edition, 1988.Google Scholar
- 4.Inmos/SGS-Thomson Microelectronics. The T9000 Transputer Product Overview, 1991.Google Scholar
- 5.K. Obermayer, H.Heller, H. Ritter, and K. Schulten. Simulation of Self-Organizing Neural Nets: a Comparison between a Transputer Ring and a Connection Machine CM-2. In Proceedings of the Third Conference of NATUG, Sunnyvale, CA, 1990.Google Scholar
- 6.D.E. Rumelhart and J.L. McClelland. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1. MIT Press, 1986.Google Scholar
- 7.L.G. Vuurpijl and Th.E. Schouten. Performance of MIMD Execution Platforms for PNNs: How many MCUPS? Technical report, Department of Real-Time Systems, Faculty of Mathematics and Informatics, University of Nijmegen, Toernooiveld 1, 6525 ED Nijmegen, The Netherlands, August 1993. In progress.Google Scholar
- 8.M. Witbrock and M. Zagha. An Implementation of Backpropagation Learning on GF11, a Large SIMD Parallel Computer. Parallel Computing, 14:329–346, 1990.Google Scholar