Implementing delay insensitive oscillatory neural networks using CMOS and emerging technology
- 358 Downloads
One major challenge in efficiently implementing neuromorphic networks is the need for a large number of variable synaptic connections. Networks that use emerging resistive memories as synapses have been proposed to tackle this problem, but interfacing with these devices is still inefficient in deeply-scaled CMOS. Oscillatory Neural Networks (ONNs) use a different paradigm than most analog hardware implementations, and may be able to interface more efficiently with RRAM neurons. Previous work on ONNs, however, has not considered the effects of actual hardware implementation realities, such as delay in the network. In this work, the first reported IC implementation of an oscillatory neural network is designed and fabricated. Modifications are made to the ONN architecture based on theoretical analysis to allow for proper operation in real-world conditions. One modification is changing the PLL-type, giving the system a different dynamic trajectory which is robust to global delays. Additionally, circuitry is added to control the transport delay of the neuron output signals. A chip with the modified ONN architecture is designed and tested in 28 nm CMOS and estimated power and area figures are reported.
KeywordsNeuromorphic computing Oscillatory neural network RRAM crossbar synapses
Thanks to George Bocchetti and Lily Zhang for assistance in the design and simulation of various PLL components.
- 1.Burr, G. W., Shelby, R. M., Sidler, S., Di Nolfo, C., Jang, J., Boybat, I., et al. (2015). Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element. Electron Devices, IEEE Transactions on, 62(11), 3498–3507.CrossRefGoogle Scholar
- 2.Goda, A.: Recent progress and future directions in nand flash scaling. In: Non-Volatile Memory Technology Symposium (NVMTS), 2013 13th, pp. 1–4. IEEE (2013)Google Scholar
- 3.Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1764–1772 (2014)Google Scholar
- 6.Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 1725–1732. IEEE (2014)Google Scholar
- 8.Lu, J., Young, S., Arel, I., Holleman, J. (2015). A 1 TOPS/W analog deep machine-learning engine with floating-gate storage in 0.13 \(\mu\)m CMOS. Solid-state circuits. IEEE Journal of 50(1), 270–281Google Scholar
- 9.Prezioso, M., Merrikh-Bayat, F., Chakrabarti, B., Strukov, D. (2016). RRAM-based hardware implementations of artificial neural networks: progress update and challenges ahead. In: SPIE OPTO (pp. 974,918–974,918). International Society for Optics and Photonics Google Scholar
- 10.Razavi, B. (1998). RF Microelectronics, vol. 1. Prentice Hall New JerseyGoogle Scholar
- 12.Schemmel, J., Fieres, J., Meier, K. (2008). Wafer-scale integration of analog neural networks. In: Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, pp. 431–438. IEEEGoogle Scholar
- 13.Shi, R., Jackson, T. C., Swenson, B., Kar, S., Pileggi, L. (2016). On the design of phase locked loop oscillatory neural networks: mitigation of transmission delay effects. In: Neural Networks, 2016. IJCNN 2016. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on. IEEEGoogle Scholar