Advertisement

Object oriented design of a simulator for large BP Neural Networks

  • J. M. Adamo
  • D. Anguita
Neurosimulators
Part of the Lecture Notes in Computer Science book series (LNCS, volume 930)

Abstract

In this paper we describe the implementation of the backpropagation algorithm by means of an object oriented library (ARCH). The use of this library relieves the user from the details of a specific parallel programming machines and at the same time allows a greater portability of the generated code.

To provide a comparison with existing solutions, we survey the most relevant implementations of the algorithm proposed so far in the literature, both on dedicated and general purpose computers.

Extensive experimental results show that the use of the library does not hurt the performance of our simulator, on the contrary our implementation on a Connection Machine (CM-5) is comparable with the fastest in its category.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    SNAP — SIMD Numerical Array Processor. HNC, 5930 Cornerstone Court West, S.Diego, CA, 1994.Google Scholar
  2. [2]
    J.M.Adamo. Object-Oriented Parallel Programming: Design and Development of an Object-Oriented Library for SPMD Programming. ICSI Technical Report TR-94-011, February 1994.Google Scholar
  3. [3]
    J.M.Adamo. Development of Parallel BLAS with ARCH Object-Oriented Parallel Library, Implementation on CM-5. ICSI Technical Report TR-94-045, August 1994.Google Scholar
  4. [4]
    D.Anguita, G.Parodi, and R.Zunino. An Efficient Implementation of BP on RISC-based Workstations. Neurocomputing 6, 1994, pp. 57–65.Google Scholar
  5. [5]
    K.Asanović, J.Beck, T.Callahan, J.Feldman, B.Irissou, B.Kingsbury, P.Kohn, J.Lazzaro, N.Morgan, D.Stoutamire and J.Wawrzynek. CNS-1 Architecture Specification. ICSI Technical Report TR-93-021, April 1993.Google Scholar
  6. [6]
    K.Asanović, J.Beck, B.E.D.Kingsbury, P.Kohn, N.Morgan, J.Wawrzynek. SPERT: A VLIW/SIMD Microprocessor for Artificial Neural Network Computations. ICSI Tech. Rep. TR-91-072, January 1992.Google Scholar
  7. [7]
    A.Corana, C.Rolando, S.Ridella. A Highly Efficient Implementation of Back-propagation Algorithm on SIMD Computers. In High Performance Computing, J.-L.Delhaye and E.Gelenbe (Eds.), Elsevier, 1989, pp.181–190.Google Scholar
  8. [8]
    A.Corana, C.Rolando, S.Ridella. Use of Level 3 BLAS Kernels in Neural Networks: The Back-propagation algorithm. Parallel Computing 89, 1990, pp.269–274.Google Scholar
  9. [9]
    J.Dongarra. Linear Algebra Library for High-Performance Computers. Frontiers of Supercomputing II. K.R.Ames and A.Brenner (Eds.), University of California Press, 1994.Google Scholar
  10. [10]
    B.Faure, G.Mazare. Implementation of back-propagation on a VLSI asynchronous cellular architecture. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, pp. 631–634.Google Scholar
  11. [11]
    K.A.Grajski, G.Chinn, C.Chen, C.Kuszmaul, S.Tomboulian. Neural Network Simulation on the MasPar MP-1 Massively Parallel Processor. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, p.673.Google Scholar
  12. [12]
    A.Hiraiwa, S.Kurosu, S.Arisawa, M.Inoue. A two level pipeline RISC processor array for ANN. Proc. of the Int. Joint Conf. on NN, January 15–19, 1990, Washinghton, DC, pp.II137–II140.Google Scholar
  13. [13]
    P.Ienne. Architectures for Neuro-Computers: Review and Performance Evaluation. Technical Report 93/21, Swiss Federal Institute of Technology, Lausanne, January 1993.Google Scholar
  14. [14]
    D.Jackson, D.Hammerstrom. Distributing Back Propagation Networks Over the Intel iPSC/860 Hypercube. Proc. of the Int. Joint Conf. on NN, July 8–12, 1991, Seattle, WA, pp. 1569–1574.Google Scholar
  15. [15]
    H.Kato, H.Yoshizawa, H.Iciki, K.Asakawa. A Parallel Neurocomputer Architecture towards Billion Connection Update Per Second. Proc. of the Int. Joint Conf. on NN, January 15–19, 1990, Washinghton, DC, pp.II47–II50.Google Scholar
  16. [16]
    X.Liu and G.L.Wilcox. Benchmarking of the CM-5 and the Cray Machines with a Very Large Backpropagation Neural Network. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp.22–27.Google Scholar
  17. [17]
    R.Means, L.Lisenbee. Extensible Linear Floating Point SIMD Neurocomputer Array Processor. Proc. of the Int. Joint Conf. on NN, July 8–12, 1991, Seattle, WA, pp. 1587–1592.Google Scholar
  18. [18]
    M.Moller. Supervised Learning on Large Redundant Training Sets. Int. J. of Neural Systems, Vol.4, No.1, 1993.Google Scholar
  19. [19]
    N.Morgan, J.Beck, P.Kohn, J.Bilmes, E.Allman, J.Beer. The Ring Array Processor: A Multiprocessing Peripheral for Connectionist Applications. Journal of Parallel and Distributed Computing, Vol.14, N.3, March 1992, pp.248–259.Google Scholar
  20. [20]
    U.A.Müller. A High Performance Neural Net Simulation Environment. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp.1–4.Google Scholar
  21. [21]
    U.A.Müller, M.Kocheisen, A.Gunzinger. High-Performance Neural Net Simulation on a Multiprocessor System with “Intelligent” Communication. Advances in Neural Information Processing Systems 6, J.D.Cowan, G.Tesauro, J.Alspector (Eds.), Morgan Kaufmann Publ., 1994, pp.888–895.Google Scholar
  22. [22]
    S.M.Müller. A Performance Analysis of the CNS-1 on Large, Dense Backpropagation Networks. ICSI Technical Report TR-93-046, September 1993.Google Scholar
  23. [23]
    A.Petrowsky, G.Dreyfus, Performance Analysis of a Pipelined Backpropagation Parallel Algorithm. IEEE Trans. on Neural Networks, Vol.4, No.6, Nov. 1993, pp.970–981.Google Scholar
  24. [24]
    U. Ramacher. SYNAPSE — A Neurocomputer That Synthesizes Neural Algorithms on a Parallel Systolic Engine. Journal of Parallel and Distributed Computing, Vol.14, N.3, March 1992, pp.306–318.CrossRefGoogle Scholar
  25. [25]
    E.Sànchez, S.Barro, C.V.Regueiro. Artificial Neural Networks Implementation on Vectorial Supercomputers. Proc. of IEEE Int. Conf. on NN, June 28–July 2, 1994, Orlando, FL, pp. 3938–3943.Google Scholar
  26. [26]
    T.J.Sejnowsky and C.R.Rosenberg. Parallel Networks that Learn to Pronounce English Text. Complex Systems 1, 1987.Google Scholar
  27. [27]
    A.Singer. Exploiting the Inherent Parallelism of Artificial Neural Networks to Achieve 1300 Million Interconnets per Second. Proc. of Int. NN Conf., July 9–13, 1990, Paris, France, pp.656–660.Google Scholar
  28. [28]
    M.A.Viredaz. MANTRA I: An SIMD Processor Array for Neural Computation. Proc. of the Euro-ARCH '93 Conf., München, October 1993.Google Scholar
  29. [29]
    J.Wawrzynek, K.Asanović, and N.Morgan. The Design of a Neuro-Microprocessor. IEEE Trans. on NN, Vol.4, No.3, May 1993.Google Scholar
  30. [30]
    X.Zhang, M.Mckenna, J.P.Mesirov, D.L.Waltz. An Efficient Implementation of the Back-Propagation Algorithm on the Connection Machine CM-2. Advances in Neural Information Processing Systems 2, D.S.Touretzky (Ed.), Morgan Kaufmann Publ., 1990, pp.801–809.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • J. M. Adamo
    • 1
  • D. Anguita
    • 2
  1. 1.Université Claude Bernard & LISA CPE-LyonLyon cedex 02France
  2. 2.D.I.B.E.University of GenovaGenovaItaly

Personalised recommendations