Memristor-based neural networks with weight simultaneous perturbation training
- 87 Downloads
The training of neural networks involves numerous operations on the weight matrix. If neural networks are implemented in hardware, all weights will be updated in parallel. However, neural networks based on CMOS technology face many challenges in the updating phase of weights. For example, derivation of the activation function and error back propagation make it difficult to be realized at the circuit level, even though the back propagation algorithm is rather efficient and popular in neural networks. In this paper, a novel synaptic unit based on double identical memristors is designed, on the basis of which a new neural network circuit architecture is proposed. The whole network is trained by a hardware-friendly weight simultaneous perturbation (WSP) algorithm. The hardware implementation of neural networks based on WSP algorithm only involves the feedforward circuit and does not require the bidirectional circuit. Furthermore, two forward calculations are merely needed to update all weight matrices for each pattern, which significantly simplifies the weight update circuit and allows simpler and easier implementation of the neural network in hardware. The practicability, utility and simplicity of this scheme are demonstrated by the supervised learning tasks.
KeywordsMemristor Synaptic unit Hardware implementation Multilayer neural networks (MNNs) Weight simultaneous perturbation (WSP)
This work was supported in part by the National Natural Science Foundation of China (No. 61571185), the Natural Science Foundation of Hunan Province, China (No. 2016JJ2030) and the Open Fund Project of Key Laboratory in Hunan Universities (No. 15K027).
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 1.Tao, F., Busso, C.: Gating neural network for large vocabulary audiovisual speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. (TASLP) 26(7), 1286–1298 (2018)Google Scholar
- 33.Alspector, J., Meir, R., Yuhas, B., Jayakumar, A., Lippe, D.: A parallel gradient descent method for learning in analog VLSI neural networks. In: Advances in Neural Information Processing Systems, pp. 836–844 (1993)Google Scholar
- 35.Belli, M.R., Conti, M., Turchetti, C.: Analog Brownian weight movement for learning of artificial neural networks. In: European Symposium on Artificial Neural Networks (ESANN), pp. 19–21 (1995)Google Scholar
- 36.Conti, M., Orcioni, S., Turchetti, C.: A new stochastic learning algorithm for analog hardware implementation. In: International Conference on Artificial Neural Networks (ICANN), pp. 1171–1176 (1998)Google Scholar
- 39.Bache, K., Lichman, M.: UCI machine learning repository. http://archive.ics.uci.edu/ml (2018)