Skip to main content
Book cover

Advances in Neuromorphic Hardware Exploiting Emerging Nanoscale Devices pp 129–149Cite as

Nonvolatile Memory Crossbar Arrays for Non-von Neumann Computing

Part of the Cognitive Systems Monographs book series (COSMOS,volume 31)

Abstract

In the conventional von Neumann (VN) architecture, data—both operands and operations to be performed on those operands—makes its way from memory to a dedicated central processor. With the end of Dennard scaling and the resulting slowdown in Moore’s law, the IT industry is turning its attention to non-Von Neumann (non-VN) architectures, and in particular, to computing architectures motivated by the human brain. One family of such non-VN computing architectures is artificial neural networks (ANNs). To be competitive with conventional architectures, such ANNs will need to be massively parallel, with many neurons interconnected using a vast number of synapses, working together efficiently to compute problems of significant interest. Emerging nonvolatile memories, such as phase-change memory (PCM) or resistive memory (RRAM), could prove very helpful for this, by providing inherently analog synaptic behavior in densely packed crossbar arrays suitable for on-chip learning. We discuss our recent research investigating the characteristics needed from such nonvolatile memory elements for implementation of high-performance ANNs. We describe experiments on a 3-layer perceptron network with 164,885 synapses, each implemented using 2 NVM devices. A variant of the backpropagation weight update rule suitable for NVM+selector crossbar arrays is shown and implemented in a mixed hardware–software experiment using an available, non-crossbar PCM array. Extensive tolerancing results are enabled by precise matching of our NN simulator to the conditions of the hardware experiment. This tolerancing shows clearly that NVM-based neural networks are highly resilient to random effects (NVM variability, yield, and stochasticity), but highly sensitive to gradient effects that act to steer all synaptic weights. Simulations of ANNs with both PCM and non-filamentary bipolar RRAM based on Pr\(_{1-x}\)Ca\(_x\)MnO\(_3\) (PCMO) are also discussed. PCM exhibits smooth, slightly nonlinear partial-SET (conductance increase) behavior, but the asymmetry of its abrupt RESET introduces difficulties; in contrast, PCMO offers continuous conductance change in both directions, but exhibits significant nonlinearities (degree of conductance change depends strongly on absolute conductance). The quantitative impacts of these issues on ANN performance (classification accuracy) are discussed.

Keywords

  • Synaptic Weight
  • Nonvolatile Memory
  • Weight Update
  • Nonvolatile Memory Device
  • Crossbar Array

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-81-322-3703-7_7
  • Chapter length: 21 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-81-322-3703-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

References

  1. https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#Tesla (2015)

  2. Breitwisch, M., Nirschl, T., Chen, C.F., Zhu, Y., Lee, M.H., Lamorey, M., Burr, G.W., Joseph, E., Schrott, A., Philipp, J.B., Cheek, R., Happ, T.D., Chen, S.H., Zaidi, S., Flaitz, P., Bruley, J., Dasaka, R., Rajendran, B., Rossnagel, S., Yang, M., Chen, Y.C., Bergmann, R., Lung, H.L., Lam, C.: Novel lithography–independent pore phase change memory. In: Symposium on VLSI Technology, pp. 100–101 (2007)

    Google Scholar 

  3. Burr, G.W., Narayanan, P., Shelby, R.M., Sidler, S., Boybat, I., di Nolfo, C., Leblebici, Y.: Large–scale neural networks implemented with nonvolatile memory as the synaptic weight element: comparative performance analysis (accuracy, speed, and power). In: IEDM Technical Digest, p. 4.4 (2015)

    Google Scholar 

  4. Burr, G.W., Shelby, R.M., di Nolfo, C., Jang, J., Shenoy, R., Narayanan, P., Virwani, K., Giacometti, E., Kurdi, B., Hwang, H.: Experimental demonstration and tolerancing of a large–scale neural network (165,000 synapses), using phase–change memory as the synaptic weight element. In: IEDM Technical Digest, p. 29.5 (2014)

    Google Scholar 

  5. Burr, G.W., Shelby, R.M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R.S., Narayanan, P., Virwani, K., Giacometti, E.U., Kurdi, B., Hwang, H.: Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element. IEEE Trans. Electr. Devices 62(11), 3498–3507 (2015)

    CrossRef  Google Scholar 

  6. Gupta, S., Kaldewey, T.: Private communication (2015)

    Google Scholar 

  7. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504 (2006)

    CrossRef  MathSciNet  MATH  Google Scholar 

  8. Jackson, B.L., Rajendran, B., Corrado, G.S., Breitwisch, M., Burr, G.W., Cheek, R., Gopalakrishnan, K., Raoux, S., Rettner, C.T., Padilla, A., Schrott, A.G., Shenoy, R.S., Kurdi, B.N., Lam, C.H., Modha, D.S.: Nanoscale electronic synapses using phase change devices. ACM J. Emerg. Technol. Comput. Syst. 9(2), 12 (2013)

    CrossRef  Google Scholar 

  9. Jang, J.W., Park, S., Burr, G.W., Hwang, H., Jeong, Y.H.: Optimization of conductance change in Pr\(_{1-x}\)Ca\(_x\)MnO\(_3\)-based synaptic devices for neuromorphic systems. IEEE Electr. Device Lett. 36(5), 457–459 (2015)

    CrossRef  Google Scholar 

  10. Jo, S.H., Chang, T., Ebong, I., Bhadviya, B.B., Mazumder, P., Lu, W.: Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10(4), 1297–1301 (2010)

    CrossRef  Google Scholar 

  11. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278 (1998)

    CrossRef  Google Scholar 

  12. Lee, J., Jo, M., Seong, D.J., Shin, J., Hwang, H.: Materials and process aspect of cross-point RRAM (invited). Microelectron. Eng. 88(7), 1113–1118 (2011)

    CrossRef  Google Scholar 

  13. Park, S., Kim, H., Choo, M., Noh, J., Sheri, A., Jung, S., Seo, K., Park, J., Kim, S., Lee, W., Shin, J., Lee, D., Choi, G., Woo, J., Cha, E., Jang, J., Park, C., Jeon, M., Lee, B., Lee, B., Hwang, H.: RRAM–based synapse for neuromorphic system with pattern recognition function. In: IEDM Technical Digest, p. 10.2 (2012)

    Google Scholar 

  14. Rajendran, B., Liu, Y., Seo, J.S., Gopalakrishnan, K., Chang, L., Friedman, D.J., Ritter, M.B.: Specifications of nanoscale devices and circuits for neuromorphic computational systems. IEEE Trans. Electr. Devices 60(1), 246–253 (2013)

    CrossRef  Google Scholar 

  15. Rumelhart, D., Hinton, G.E., McClelland, J.L.: A general framework for parallel distributed processing. In: Parallel Distributed Processing. MIT Press (1986)

    Google Scholar 

  16. Suri, M., Bichler, O., Querlioz, D., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., DeSalvo, B.: Phase change memory as synapse for ultra–dense neuromorphic systems: application to complex visual pattern extraction. In: IEDM Technical Digest, p. 4.4 (2011)

    Google Scholar 

  17. Suri, M., Bichler, O., Querlioz, D., Palma, G., Vianello, E., Vuillaume, D., Gamrat, C., DeSalvo, B.: CBRAM devices as binary synapses for low–power stochastic neuromorphic systems: auditory (cochlea) and visual (retina) cognitive processing applications. In: IEDM Technical Digest, p. 10.3 (2012)

    Google Scholar 

  18. Yu, S., Gao, B., Fang, Z., Yu, H., Kang, J., Wong, H.S.P.: A neuromorphic visual system using RRAM synaptic devices with sub–pj energy and tolerance to variability: experimental characterization and large–scale modeling. In: IEDM Technical Digest, p. 10.4 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. EPFL, Lausanne, Switzerland

    Severin Sidler & Irem Boybat

  2. Pohang University of Science and Technology, Pohang, Korea

    Jun-Woo Jang & Hyunsang Hwang

  3. IBM Research, Almaden, San Jose, CA, USA

    Geoffrey W. Burr, Robert M. Shelby, Carmelo di Nolfo, Pritish Narayanan & Kumar Virwani

Authors
  1. Severin Sidler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun-Woo Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Geoffrey W. Burr
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert M. Shelby
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Irem Boybat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carmelo di Nolfo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pritish Narayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kumar Virwani
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Hyunsang Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geoffrey W. Burr .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi, Delhi, India

    Manan Suri

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer (India) Pvt. Ltd.

About this chapter

Cite this chapter

Sidler, S. et al. (2017). Nonvolatile Memory Crossbar Arrays for Non-von Neumann Computing. In: Suri, M. (eds) Advances in Neuromorphic Hardware Exploiting Emerging Nanoscale Devices. Cognitive Systems Monographs, vol 31. Springer, New Delhi. https://doi.org/10.1007/978-81-322-3703-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-81-322-3703-7_7

  • Published:

  • Publisher Name: Springer, New Delhi

  • Print ISBN: 978-81-322-3701-3

  • Online ISBN: 978-81-322-3703-7

  • eBook Packages: EngineeringEngineering (R0)