Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Artificial neural networks based on memristive devices

  • 369 Accesses

  • 4 Citations

Abstract

The advent of memristive devices and the continuing research and development in the field of neuromorphic computing show great potential as an alternative to traditional von Neumann computing and bring us ever closer to realizing a true “thinking machine”. Novel neural network architectures and algorithms inspired by the brain are becoming more and more attractive to keep up with computing needs, relying on intrinsic parallelism and reduced power consumption to outperform more conventional computing methods. This article provides an overview of various neural networks with an emphasis on networks based on memristive emerging devices, with the advantages of memristor neural networks compared with pure complementary metal oxide semiconductor (CMOS) implementations. A general description of neural networks is presented, followed by a survey of prominent CMOS networks, and finally networks implemented using emerging memristive devices are discussed, along with the motivation for developing memristor based networks and the computational potential these networks possess.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Mack C A. Fifty years of Moore’s law. IEEE Trans Semicond Manufact, 2011, 24: 202–207

  2. 2

    Schulz M. The end of the road for silicon? Nature, 1999, 399: 729–730

  3. 3

    von Neumann J. First draft of a report on the EDVAC. IEEE Ann Hist Comput, 1993, 15: 11–21

  4. 4

    Anderson H C. Neural network machines. IEEE Potentials, 1989, 8: 13–16

  5. 5

    Squire L R, Berg D, Bloom F, et al. Fundamental neuroscience. Curr Opin Neurobiol, 2008, 10: 649–654

  6. 6

    Pedretti G, Milo V, Ambrogio S, et al. Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity. Sci Rep, 2017, 7: 5288

  7. 7

    Zamarreño-Ramos C, Camuñas-Mesa L A, Pérez-Carrasco J A, et al. On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front Neurosci-Switz, 2011, 5: 1–22

  8. 8

    LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436–444

  9. 9

    Li C, Hu M, Li Y, et al. Analogue signal and image processing with large memristor crossbars. Nat Electron, 2018, 1: 52–59

  10. 10

    Hebb D O. The first stage of perception: growth of the assembly BT — the organization of behavior. Organ Behav, 1949, 4: 60–78

  11. 11

    Wang Z, Joshi S, Savel’ev S E, et al. Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing. Nat Mater, 2017, 16: 101–108

  12. 12

    Hu M, Strachan J P, Li Z, et al. Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. In: Proceedings of the 53rd ACM/EDAC/IEEE Design Automation Conference (DAC), Austin, 2016. 1–6

  13. 13

    Shafiee A, Nag A, Muralimanohar N, et al. ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 14–26

  14. 14

    Chi P, Li S, Xu C, et al. PRIME: a novel processing-in-memory architecture for neural network computation in reRAM-based main memory. In: Proceedings of ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, 2016. 27–39

  15. 15

    Tang S, Yin S, Zheng S, et al. AEPE: an area and power efficient RRAM crossbar-based accelerator for deep CNNs. In: Proceedings of IEEE 6th Non-Volatile Memory Systems and Applications Symposium (NVMSA), Hsinchu, 2017. 1–6

  16. 16

    Yao P, Wu H, Gao B, et al. Face classification using electronic synapses. Nat Commun, 2017, 8: 15199

  17. 17

    Wang Z, Joshi S, Savel’ev S, et al. Fully memristive neural networks for pattern classification with unsupervised learning. Nat Electron, 2018, 1: 137–145

  18. 18

    Sheridan P M, Cai F, Du C, et al. Sparse coding with memristor networks. Nat Nanotech, 2017, 12: 784–789

  19. 19

    Hu M, Graves C E, Li C, et al. Memristor-based analog computation and neural network classification with a dot product engine. Adv Mater, 2018, 30: 1705914

  20. 20

    Li C, Belkin D, Li Y, et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat Commun, in press, 2018

  21. 21

    Zarudnyi K, Mehonic A, Montesi L, et al. Spike-timing dependent plasticity in unipolar silicon oxide RRAM devices. Front Neurosci, 2018, 12: 57

  22. 22

    Yu S M, Li Z W, Chen P-Y, et al. Binary neural network with 16 Mb RRAM macro chip for classification and online training. In: Proceedings of IEEE International Electron Devices Meeting (IEDM), San Francisco, 2016. 1–4

  23. 23

    Prezioso M, Merrikh-Bayat F, Hoskins B D, et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 2015, 521: 61–64

  24. 24

    Yu S. Neuro-inspired computing with emerging nonvolatile memorys. Proc IEEE, 2018, 106: 260–285

  25. 25

    Alibart F, Zamanidoost E, Strukov D B, et al. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat Commun, 2013, 2013: 2072

  26. 26

    Merrikh B F, Prezioso M, Chakrabarti B, et al. Advancing memristive analog neuromorphic networks: increasing complexity, and coping with imperfect hardware components. ArXiv: 1611.04465

  27. 27

    Du C, Cai F, Zidan M A, et al. Reservoir computing using dynamic memristors for temporal information processing. Nat Commun, 2017, 8: 2204

  28. 28

    Mehonic A, Kenyon A J. Emulating the electrical activity of the neuron using a silicon oxide RRAM cell. Front Neurosci-Switz, 2016, 10: 57

  29. 29

    Chen Y J, Luo T, Liu S L, et al. DaDianNao: a machine-learning supercomputer. In: Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture, Cambridge, 2014. 609–622

  30. 30

    Merolla P A, Arthur J V, Alvarez-Icaza R, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 2014, 345: 668–673

  31. 31

    Jouppi N P, Young C, Patil N, et al. In-Datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, Toronto, 2017. 1–12

  32. 32

    Gawande N A, Landwehr J B, Daily J A, et al. Scaling deep learning workloads: NVIDIA DGX-1/Pascal and intel knights landing. In: Proceedings of 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Lake Buena Vista, 2017. 399–408

  33. 33

    Chua L O. Memristor — the missing circuit element. IEEE Trans Circuits Syst, 1971, 18: 507–519

  34. 34

    Strukov D B, Snider G S, Stewart D R, et al. The missing memristor found. Nature, 2008, 453: 80–83

  35. 35

    Yang J J, Pickett M D, Li X, et al. Memristive switching mechanism for metal/oxide/metal nanodevices. Nat Nanotech, 2008, 3: 429–433

  36. 36

    Yang J J, Miao F, Pickett M D, et al. The mechanism of electroforming of metal oxide memristive switches. Nanotechnology, 2009, 20: 215201

  37. 37

    Chua L. Resistance switching memories are memristors. Appl Phys A, 2011, 102: 765–783

  38. 38

    Yang J J, Strukov D B, Stewart D R. Memristive devices for computing. Nat Nanotech, 2013, 8: 13–24

  39. 39

    Yang J J, Williams R S. Memristive devices in computing system: promises and challenges. ACM J Emerg Tech Com, 2013, 9: 1–20

  40. 40

    Pickett M D, Strukov D B, Borghetti J L, et al. Switching dynamics in titanium dioxide memristive devices. J Appl Phys, 2009, 106: 074508

  41. 41

    Alibart F, Gao L, Hoskins B D, et al. High precision tuning of state for memristive devices by adaptable variationtolerant algorithm. Nanotechnology, 2012, 23: 075201

  42. 42

    Choi S, Sheridan P, Lu W D. Data clustering using memristor networks. Sci Rep, 2015, 5: 10492

  43. 43

    Yang J J, Zhang M X, Strachan J P, et al. High switching endurance in TaOx memristive devices. Appl Phys Lett, 2010, 97: 232102

  44. 44

    Choi B J, Torrezan A C, Strachan J P, et al. High-speed and low-energy nitride memristors. Adv Funct Mater, 2016, 26: 5290–5296

  45. 45

    Yoon J H, Zhang J, Ren X, et al. Truly electroforming-free and low-energy memristors with preconditioned conductive tunneling paths. Adv Funct Mater, 2017, 27: 1702010

  46. 46

    Li C, Han L, Jiang H, et al. Three-dimensional crossbar arrays of self-rectifying Si/SiO2/Si memristors. Nat Commun, 2017, 8: 15666

  47. 47

    Shulaker M M, Hills G, Park R S, et al. Three-dimensional integration of nanotechnologies for computing and data storage on a single chip. Nature, 2017, 547: 74–78

  48. 48

    Yu S, Wu Y, Jeyasingh R, et al. An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE Trans Electron Devices, 2011, 58: 2729–2737

  49. 49

    Chang T, Yang Y, Lu W. Building neuromorphic circuits with memristive devices. IEEE Circuits Syst Mag, 2013, 13: 56–73

  50. 50

    Chang T, Jo S H, Kim K H, et al. Synaptic behaviors and modeling of a metal oxide memristive device. Appl Phys A, 2011, 102: 857–863

  51. 51

    Gaba S, Sheridan P, Zhou J, et al. Stochastic memristive devices for computing and neuromorphic applications. Nanoscale, 2013, 5: 5872–5878

Download references

Author information

Correspondence to Qiangfei Xia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ravichandran, V., Li, C., Banagozar, A. et al. Artificial neural networks based on memristive devices. Sci. China Inf. Sci. 61, 060423 (2018). https://doi.org/10.1007/s11432-018-9425-1

Download citation

Keywords

  • neuromorphic computing
  • memristors
  • neural networks