Parallel implementations of feed-forward neural network using MPI and C# on .NET platform

  • U. Lotrič
  • A. Dobnikar
Conference paper


The parallelization of gradient descent training algorithm with momentum and the Levenberg-Marquardt algorithm is implemented using C# and Message Passing Interface (MPI) on .NET platform. The turnaround times of both algorithms are analyzed on cluster of homogeneous computers. It is shown that the optimal number of cluster nodes is a compromise between the decrease of computational time due to parallelization and corresponding increase of time needed for communication.


Message Passing Interface Single Instruction Multiple Data Gradient Descent Algorithm Weight Update Hessian Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Strey, A. (2003) On the suitability of SIMD extensions for neural network simulation. Microprocessors and Microsystems 27: 341–351CrossRefGoogle Scholar
  2. [2]
    Gropp, W., Lusk, E., Skjellum, A. (1999) Using MPI: portable parallel programming with the message-passing interface, MIT, CambridgeGoogle Scholar
  3. [3]
    ECMA (2001) Common language infrastructure (CLI), C# language specification, ECMA, http://www.ecma.chGoogle Scholar
  4. [4]
    Haykin, S. (1999) Neural networks: a comprehensive foundation, 2nd ed., Prentice-Hall, New JerseyGoogle Scholar
  5. [5]
    Hagan, M. T., Menhaj, M. B. (1994) Training feed-forward networks with the marquardt algorithm, IEEE Trans. Neural Netw 5(6): 989–993CrossRefGoogle Scholar
  6. [6]
    Mono project (2004) Open source platform based on.NET, http://www.mono-project.comGoogle Scholar
  7. [7]
    Willcock, J., et. al. (2002) Using MPI with C# and the Common language infrastructure, Technical report TR570, Indiana University, BloomingtonGoogle Scholar
  8. [8]
    Ranga Suri, N. N. R., et. al. (2002) Parallel Levenberg-Marquardt-based neural network training on Linux clusters-a case study, 3rd Indian conference on computer vision, graphics and image processing.Google Scholar

Copyright information

© Springer-Verlag/Wien 2005

Authors and Affiliations

  • U. Lotrič
    • 1
  • A. Dobnikar
    • 1
  1. 1.Faculty of Computer and Information ScienceUniversity of LjubljanaSlovenia

Personalised recommendations