Advertisement

A Circuit Architecture for Analog On-Chip Back Propagation Learning with Local Learning Rate Adaptation

  • G. M. Bo
  • D. D. Caviglia
  • H. Chible`
  • M. Valle
Article

Abstract

In this paper we present the analog CMOS architecture of a Multi Layer Perceptron network with on-chip stochastic Back Propagation learning. The learning algorithm is based on a local learning rate adaptation technique which makes the on-chip implementation more efficient (i.e. fast convergence speed) with respect to similar architectures presented in the literature. Circuit simulation results on the XOR learning problem validate the network behavior.

neural networks on-chip learning local learning rate adaptation analog circuits 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. Alspector, R. Meir, B. Yuhas, A. Jayakumar, and D. Lippe, “A Parallel Gradient Descent Method for Learning in Analog VLSI Neural Networks.” in Advances in Neural Information Processing Systems 5 (NIPS5). pp. 836–844, 1993.Google Scholar
  2. 2.
    A. J. Annema, Feed-Forward Neural Networks. Kluwer Academic Publisher, 1995.Google Scholar
  3. 3.
    Y. Berg, R. L. Sigvartsen, T. S. Lande, and A. Abusland, “An Analog Feed-Forward Neural Network with On-Chip Learning.” Analog Integrated Circuits and Signal Processing 9, pp. 65–75, 1996.Google Scholar
  4. 4.
    G. Cauwenberghs, “A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization.” in Advances in Neural Information Processing Systems 5 (NIPS5), pp. 244–251, 1993.Google Scholar
  5. 5.
    H. Chiblè, “Analysis and Design of Analog Microelectronic Neural Network Architectures with On-Chip Supervised Learning.” Ph.D. Thesis, Sep. 25, 1997, DIBE, University of Genoa.Google Scholar
  6. 6.
    B. K. Dolenko and H. C. Card, “Tolerance to Analog Hardware of On-Chip Learning in Backpropagation Networks.” IEEE Trans. on Neural Networks 6(5), pp. 1045–1052, 1995.Google Scholar
  7. 7.
    P. J. Edwards and A. F. Murray, Analog Imprecision in MLP Training. World Scientific Publishing Co. Pte. Ltd., 1996.Google Scholar
  8. 8.
    R. L. Geiger, P. E. Allen, and N. R. Strader, VLSI Design Techniques for Analog and Digital Circuits. McGraw-Hill Book Co., 1990.Google Scholar
  9. 9.
    J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of the Neural Computation. Addison-Wesley Publishing Company, 1981.Google Scholar
  10. 10.
    B. A. Jacobs, “Increased Rates of Convergence through Learning Rate Adaptation.” Neural Networks 4, pp. 295–307, 1988.Google Scholar
  11. 11.
    C. A. Mead, Analog VLSI and Neural Systems. Addison-Wesley Reading, 1989.Google Scholar
  12. 12.
    T. Morie and Y. Amemiya, “An All-Analog Expandable Neural-Network LSI with On-Chip Backpropagation Learning.” IEEE Journal of Solid State Circuits 29(9), pp. 1086–1093, 1994.Google Scholar
  13. 13.
    L. M. Reyneri and E. Filippi, “An Analysis on the Performance of Silicon Implementations of Backpropagation Algorithms for Artificial Neural Networks.” IEEE Trans. on Computers 12, pp. 1380–1389, 1991.Google Scholar
  14. 14.
    D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing. Cambridge: MIT Press, 1986.Google Scholar
  15. 15.
    T. Tollenaere, “SuperSAB: Fast Adaptative Back propagation With Good Scaling Properties.” Neural Networks 3, pp. 561–573, 1990.Google Scholar
  16. 16.
    M. Valle, D. D. Caviglia, G. Donzellini, A. Mussi, F. Oddone, and G. M. Bisio, “A Neural Computer based on an Analog VLSI Neural Network.” In Proc. of International Conference on Artificial Neural Network, ICANN94, 2, pp. 1339–1342, 1994.Google Scholar
  17. 17.
    M. Valle, D. D. Caviglia, and G. M. Bisio, “An Analog VLSI Neural Network with On-Chip Back Propagation Learning.” Analog Integrated Circuits and Signal Processing. Kluwer Academic Publisher, 9, pp. 231–245, 1996.Google Scholar
  18. 18.
    E. A. Vittoz, “Analog VLSI Signal Processing: Why, Where and How.” Journal of VLSI Signal Processing 8, pp. 27–44, 1994.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • G. M. Bo
    • 1
  • D. D. Caviglia
    • 1
  • H. Chible`
    • 1
  • M. Valle
    • 1
  1. 1.Department of Biophysical and Electronic EngineeringUniversity of GenoaGenovaItaly

Personalised recommendations