Parallel implementations of feed-forward neural network using MPI and C# on .NET platform
The parallelization of gradient descent training algorithm with momentum and the Levenberg-Marquardt algorithm is implemented using C# and Message Passing Interface (MPI) on .NET platform. The turnaround times of both algorithms are analyzed on cluster of homogeneous computers. It is shown that the optimal number of cluster nodes is a compromise between the decrease of computational time due to parallelization and corresponding increase of time needed for communication.
KeywordsMessage Passing Interface Single Instruction Multiple Data Gradient Descent Algorithm Weight Update Hessian Approximation
Unable to display preview. Download preview PDF.
- Gropp, W., Lusk, E., Skjellum, A. (1999) Using MPI: portable parallel programming with the message-passing interface, MIT, CambridgeGoogle Scholar
- ECMA (2001) Common language infrastructure (CLI), C# language specification, ECMA, http://www.ecma.chGoogle Scholar
- Haykin, S. (1999) Neural networks: a comprehensive foundation, 2nd ed., Prentice-Hall, New JerseyGoogle Scholar
- Mono project (2004) Open source platform based on.NET, http://www.mono-project.comGoogle Scholar
- Willcock, J., et. al. (2002) Using MPI with C# and the Common language infrastructure, Technical report TR570, Indiana University, BloomingtonGoogle Scholar
- Ranga Suri, N. N. R., et. al. (2002) Parallel Levenberg-Marquardt-based neural network training on Linux clusters-a case study, 3rd Indian conference on computer vision, graphics and image processing.Google Scholar