Experimental Demonstration of Firing Rate Neural Networks Based on Metal-Oxide Memristive Crossbars

  • Farnood Merrikh Bayat
  • Mirko Prezioso
  • Bhaswar Chakrabarti
Chapter

Abstract

Limitations of currently dominating von Neumann architectures have pushed the research toward brain-inspired solutions like neural networks to reach new levels of computing efficiency. While these highly parallelized architectures have achieved outstanding performances at software level, their hardware implementation is still a challenging problem due to the large amount of memory and arithmetic units required to store synaptic weights and implement synaptic transmission functions. One possible solution to tackle this challenging problem is to build neural networks with analog circuits and implement the functionality of synapses with the same nonvolatile memory unit that stores synaptic weights (a so-called in-memory computing). Scalability, analog behavior (multilevel programmability), speed, and power consumption are fundamental performance requirements for such memory arrays, given the huge number of synapses in state-of-the-art neural networks. Among different choices, Resistive Random Access Memories (RRAMs) are considered as one of the main candidates due to their excellent scalability and the possibility of their integration with fully mature CMOS technology. Here in this chapter, we will show the basic principles of such devices, alongside their performance as single devices and also when integrated in crossbars, all induced from experiments. We will also present the results of the first experimental realizations of single- and multi-layer neural networks with synapses implemented through crossbar arrays with detailed explanation on their in situ training. Finally, we conclude the chapter by presenting our recent progress on the 3D integration of resistive switching devices on top of CMOS-based circuitries and also by discussing the limitations of current implementations and future challenges.

References

  1. 1.
    J. Hasler, H.B. Marr, Finding a roadmap to achieve large neuromorphic hardware systems. Front. Neurosci. 7, 118 (2013)CrossRefGoogle Scholar
  2. 2.
    C. Diorio, P. Hasler, A. Minch, C.A. Mead, A single-transistor silicon synapse. IEEE Trans Electron Devices 43(11), 1972–1980 (1996)CrossRefGoogle Scholar
  3. 3.
    S. George et al., A programmable and configurable mixed-mode FPAA SoC. IEEE Trans Very Large Scale Integr (VLSI) Syst 24(6), 2253–2261 (2016)Google Scholar
  4. 4.
    K.K. Likharev, CrossNets: neuromorphic hybrid CMOS/nanoelectronic networks. Sci. Adv. Mater. 3(3), 322–331 (2011)CrossRefGoogle Scholar
  5. 5.
    K.K. Likharev, Hybrid CMOS/nanoelectronic circuits: Opportunities and challenges. J. Nanoelectron. Optoelectron. 3(3), 203–230 (2008)CrossRefGoogle Scholar
  6. 6.
    M. Prezioso, F. Merrikh-Bayat, B.D. Hoskins, G.C. Adam, K.K. Likharev, D.B. Strukov, Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521(7550), 61–64 (2015)CrossRefGoogle Scholar
  7. 7.
    M. Prezioso, et al., Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2-x/Pt Memristors, in IEEE International Electron Devices Meeting (IEDM) (2015)Google Scholar
  8. 8.
    L.O. Chua, S.M. Kang, Memristive devices and systems. Proc. IEEE 64(2), 209–223 (1976)MathSciNetCrossRefGoogle Scholar
  9. 9.
    R. Waser, M. Aono, Nanoionics-based resistive switching memories. Nat. Mater. 6(11), 833–840 (2007)CrossRefGoogle Scholar
  10. 10.
    M.D. Pickett et al., Switching dynamics in titanium dioxide memristive devices. J. Appl. Phys. 106(7), 74508 (2009)CrossRefGoogle Scholar
  11. 11.
    J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the theory of neural computation (Westview Press, Boulder, Revised edition 1991)Google Scholar
  12. 12.
    F. Alibart, L. Gao, B.D. Hoskins, D.B. Strukov, High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology 23(7), 75201 (2012)CrossRefGoogle Scholar
  13. 13.
    M.A. Zidan, H.A.H. Fahmy, M.M. Hussain, K.N. Salama, Memristor-based memory: The sneak paths problem and solutions. Microelectron. J. 44(2), 176–183 (2013)CrossRefGoogle Scholar
  14. 14.
    R. Hecht-Nielsen, Theory of the backpropagation neural network, in International Joint Conference on Neural Networks (IJCNN) (1989), pp. 593–605Google Scholar
  15. 15.
    J.J. Yang, D.B. Strukov, D.R. Stewart, Memristive devices for computing. Nat. Nanotechnol. 8(1), 13–24 (2013)CrossRefGoogle Scholar
  16. 16.
    W. Schiffmann, M. Joost, R. Werner, Optimization of the backpropagation algorithm for training multilayer perceptrons, Univ. Koblenz, Inst. Physics, Rheinau (1992), pp. 3–4Google Scholar
  17. 17.
    I. Kataeva, F. Merrikh-Bayat, E. Zamanidoost, D. Strukov, Efficient training algorithms for neural networks based on memristive crossbar circuits, in International Joint Conference on Neural Networks (IJCNN) (2015), pp. 1–8Google Scholar
  18. 18.
    F. Alibart, E. Zamanidoost, D.B. Strukov, Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2 (2013)CrossRefGoogle Scholar
  19. 19.
    F. M. Bayat, M. Prezioso, B. Chakrabarti, I. Kataeva, and D. B. Strukov, “Advancing memristive analog neuromorphic networks: increasing complexity, and coping with imperfect hardware components,” arXiv preprint arXiv:1611.04465, 2016Google Scholar
  20. 20.
    E.B. Baum, On the capabilities of multilayer perceptrons. J. Complex. 4(3), 193–215 (1988)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  22. 22.
    A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (NIPS) (The MIT Press, Cambridge, 2012), pp. 1097–1105Google Scholar
  23. 23.
    S.-S. Sheu et al., A 4 Mb embedded SLC resistive-RAM macro with 7.2 ns read-write random-access time and 160 ns MLC-access capability, in 2011 IEEE International Solid-State Circuits Conference (ISSCC), (2011)Google Scholar
  24. 24.
    T. Liu et al., A 130.7-2-layer 32-Gb ReRAM memory device in 24-nm technology. IEEE J. Solid State Circuits 49(1), 140–153 (2014)CrossRefGoogle Scholar
  25. 25.
    Q. Xia et al., Memristor- CMOS hybrid integrated circuits for reconfigurable logic. Nano Lett. 9(10), 3640–3645 (2009)CrossRefGoogle Scholar
  26. 26.
    P. Lin, S. Pi, Q. Xia, 3D integration of planar crossbar memristive devices with CMOS substrate. Nanotechnology 25(40), 405202 (2014)CrossRefGoogle Scholar
  27. 27.
    B. Chakrabarti, et al., A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuits, submitted Google Scholar
  28. 28.
    M. A. Lastras-Montano, A. Ghofrani, K.-T. Cheng, Architecting energy efficient crossbar-based memristive random-access memories, in IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH) (2015)Google Scholar
  29. 29.
    M. Payvand et al., A configurable CMOS memory platform for 3D-integrated memristors, in IEEE International Symposium on Circuits and Systems (ISCAS) (2015), pp. 1378–1381Google Scholar
  30. 30.
    L. Zhao et al., Multi-level control of conductive nano-filament evolution in HfO 2 ReRAM by pulse-train operations. Nanoscale 6(11), 5698–5702 (2014)CrossRefGoogle Scholar
  31. 31.
    W. Kim et al., Forming-free nitrogen-doped AlOx RRAM with sub-μA programming current, in Symposium on VLSI Technology (2011)Google Scholar
  32. 32.
    J.J. Yang et al., Engineering nonlinearity into memristors for passive crossbar applications. Appl. Phys. Lett. 100(11), 113501 (2012)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Y. Huang et al., A new dynamic selector based on the bipolar RRAM for the crossbar array application. IEEE Trans Electron Devices 59(8), 2277–2280 (2012)CrossRefGoogle Scholar
  34. 34.
    J. Woo, D. Lee, E. Cha, S. Lee, S. Park, H. Hwang, Vertically stacked ReRAM composed of a bidirectional selector and CB-RAM for cross-point array applications. IEEE Electron Device Lett 34(12), 1512–1514 (2013)CrossRefGoogle Scholar
  35. 35.
    S. H. Jo, T. Kumar, M. Asnaashari, W. D. Lu, H. Nazarian, 3D ReRAM with field assisted super-linear threshold (FAST) selector technology for super-dense, low power, low latency data storage systems, in IEEE/ACM Asia and South Pacific Design Automation Conference (ASP-DAC) (2015), pp. 575–575Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Farnood Merrikh Bayat
    • 1
  • Mirko Prezioso
    • 1
  • Bhaswar Chakrabarti
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of California Santa BarbaraSanta BarbaraUSA

Personalised recommendations