Analog Integrated Circuits and Signal Processing

, Volume 89, Issue 3, pp 619–629 | Cite as

Implementing delay insensitive oscillatory neural networks using CMOS and emerging technology

  • Thomas C. JacksonEmail author
  • Rongye Shi
  • Abhishek A. Sharma
  • James A. Bain
  • Jeffrey A. Weldon
  • Lawrence Pileggi


One major challenge in efficiently implementing neuromorphic networks is the need for a large number of variable synaptic connections. Networks that use emerging resistive memories as synapses have been proposed to tackle this problem, but interfacing with these devices is still inefficient in deeply-scaled CMOS. Oscillatory Neural Networks (ONNs) use a different paradigm than most analog hardware implementations, and may be able to interface more efficiently with RRAM neurons. Previous work on ONNs, however, has not considered the effects of actual hardware implementation realities, such as delay in the network. In this work, the first reported IC implementation of an oscillatory neural network is designed and fabricated. Modifications are made to the ONN architecture based on theoretical analysis to allow for proper operation in real-world conditions. One modification is changing the PLL-type, giving the system a different dynamic trajectory which is robust to global delays. Additionally, circuitry is added to control the transport delay of the neuron output signals. A chip with the modified ONN architecture is designed and tested in 28 nm CMOS and estimated power and area figures are reported.


Neuromorphic computing Oscillatory neural network RRAM crossbar synapses 



Thanks to George Bocchetti and Lily Zhang for assistance in the design and simulation of various PLL components.


  1. 1.
    Burr, G. W., Shelby, R. M., Sidler, S., Di Nolfo, C., Jang, J., Boybat, I., et al. (2015). Experimental demonstration and tolerancing of a large-scale neural network (165 000 synapses) using phase-change memory as the synaptic weight element. Electron Devices, IEEE Transactions on, 62(11), 3498–3507.CrossRefGoogle Scholar
  2. 2.
    Goda, A.: Recent progress and future directions in nand flash scaling. In: Non-Volatile Memory Technology Symposium (NVMTS), 2013 13th, pp. 1–4. IEEE (2013)Google Scholar
  3. 3.
    Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1764–1772 (2014)Google Scholar
  4. 4.
    Hoppensteadt, F. C., & Izhikevich, E. M. (2000). Pattern recognition via synchronization in phase-locked loop neural networks. IEEE Transactions on Neural Networks, 11(3), 734–738.CrossRefGoogle Scholar
  5. 5.
    Jackson, T. C., Sharma, A. A., Bain, J. A., Weldon, J. A., & Pileggi, L. (2015). Oscillatory neural networks based on TMO nano-oscillators and multi-level RRAM cells. Emerging and Selected Topics in Circuits and Systems, IEEE Journal on, 5(2), 230–241.CrossRefGoogle Scholar
  6. 6.
    Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 1725–1732. IEEE (2014)Google Scholar
  7. 7.
    Linan, G., Espejo, S., Domínguez-Castro, R., & Rodríguez-Vázquez, A. (2002). ACE4k: An analog I/O 64 × 64 visual microprocessor chip with 7-bit analog accuracy. International Journal of Circuit Theory and Applications, 30(2–3), 89–116.CrossRefzbMATHGoogle Scholar
  8. 8.
    Lu, J., Young, S., Arel, I., Holleman, J. (2015). A 1 TOPS/W analog deep machine-learning engine with floating-gate storage in 0.13 \(\mu\)m CMOS. Solid-state circuits. IEEE Journal of 50(1), 270–281Google Scholar
  9. 9.
    Prezioso, M., Merrikh-Bayat, F., Chakrabarti, B., Strukov, D. (2016). RRAM-based hardware implementations of artificial neural networks: progress update and challenges ahead. In: SPIE OPTO (pp. 974,918–974,918). International Society for Optics and Photonics Google Scholar
  10. 10.
    Razavi, B. (1998). RF Microelectronics, vol. 1. Prentice Hall New JerseyGoogle Scholar
  11. 11.
    Razavi, B. (2015). The StrongARM latch [a circuit for all seasons]. Solid-State Circuits Magazine, IEEE, 7(2), 12–17.MathSciNetCrossRefGoogle Scholar
  12. 12.
    Schemmel, J., Fieres, J., Meier, K. (2008). Wafer-scale integration of analog neural networks. In: Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, pp. 431–438. IEEEGoogle Scholar
  13. 13.
    Shi, R., Jackson, T. C., Swenson, B., Kar, S., Pileggi, L. (2016). On the design of phase locked loop oscillatory neural networks: mitigation of transmission delay effects. In: Neural Networks, 2016. IJCNN 2016. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on. IEEEGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Thomas C. Jackson
    • 1
    Email author
  • Rongye Shi
    • 1
  • Abhishek A. Sharma
    • 1
  • James A. Bain
    • 1
  • Jeffrey A. Weldon
    • 1
  • Lawrence Pileggi
    • 1
  1. 1. Carnegie Mellon University PittsburghUSA

Personalised recommendations