Advertisement

Ex-situ training of large memristor crossbars for neural network applications

  • Raqibul Hasan
  • Chris Yakopcic
  • Tarek M. Taha
Article
  • 35 Downloads

Abstract

Memristor crossbar arrays carry out multiply–add operations in parallel in the analog domain, and can enable neuromorphic systems with high throughput at low energy and area consumption. Neural networks need to be trained prior to use. This work considers ex-situ training where the weights pre-trained by a software implementation are then programmed into the hardware. Existing ex-situ training approaches for memristor crossbars do not consider sneak path currents, and they may work only for neural networks implemented using small crossbars. Ex-situ training in large crossbars, without considering sneak paths, reduces the application recognition accuracy significantly due to the increased number of sneak current paths. This paper proposes ex-situ training approaches for both 0T1M and 1T1M crossbars that account for crossbar sneak paths and the stochasticity inherent in memristor switching. To carry out the simulation of these training approaches, a framework for fast and accurate simulation of large memristor crossbars was developed. The results in this work show that 0T1M crossbar based systems can be 17–83% smaller in area than 1T1M crossbar based systems.

Keywords

Memristor crossbars Neural networks Training Ex-situ training Sneak paths Low power pattern recognition system 

Notes

Acknowledgement

This work was supported in part by a CAREER grant from the US National Science Foundation under Grant 1053149.

References

  1. 1.
    Larochelle, H., Bengio, Y., Louradour, J., & Lamblin, P. (2009). Exploring strategies for training deep neural networks. Journal of Machine Learning Research, 10, 1–40.zbMATHGoogle Scholar
  2. 2.
    Chua, L. O. (1971). Memristor—The missing circuit element. IEEE Transactions on Circuit Theory, 18(5), 507–519.CrossRefGoogle Scholar
  3. 3.
    Alibart, F., Zamanidoost, E., & Strukov, D. B. (2013). Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nature Communications, 4, 2072.CrossRefGoogle Scholar
  4. 4.
    Taha, T. M., Hasan, R., & Yakopcic, C. (2014). Memristor crossbar based multicore neuromorphic processors. In IEEE international system-on-chip conference (SOCC) (pp. 383–389).Google Scholar
  5. 5.
    Soudry, D., Di Castro, D., Gal, A., Kolodny, A., & Kvatinsky, S. (2015). Memristor-based multilayer neural networks with online gradient descent training. IEEE Transactions on Neural Networks and Learning Systems, 26(10), 2408–2421.MathSciNetCrossRefGoogle Scholar
  6. 6.
    Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., & Strukov, D. B. (2015). Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature, 521(7550), 61–64.CrossRefGoogle Scholar
  7. 7.
    Yu, S., Wu, Y., & Wong, H.-S. P. (2011). Investigating the switching dynamics and multilevel capability of bipolar metal oxide resistive switching memory. Applied Physics Letters, 98, 103514.CrossRefGoogle Scholar
  8. 8.
    Dong, X., Xu, C., Member, S., Xie, Y., & Jouppi, N. P. (2012). NVSim: A circuit-level performance, energy, and area model for emerging nonvolatile memory. IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems, 31(7), 994–1007.CrossRefGoogle Scholar
  9. 9.
    Yakopcic, C., Hasan, R., & Taha, T. M. (2015). Memristor based neuromorphic circuit for ex-situ training of multi-layer neural network algorithms. In IEEE IJCNN.Google Scholar
  10. 10.
    Hasan, R., Taha, T. M., & Yakopcic, C. (2015). Ex-situ training of dense memristor crossbar for neuromorphic applications. In The proceedings of the IEEE international symposium on nanoscale architectures.Google Scholar
  11. 11.
    Yakopcic, C., Taha, T. M., & McLean, M. R. (2015). Method for ex situ training in a memristor-based neuromorphic circuit using a robust weight programming method. Electronics Letters, 51(12), 899–900.CrossRefGoogle Scholar
  12. 12.
  13. 13.
  14. 14.
  15. 15.
    Yakopcic, C., Taha, T. M., Subramanyam, G., & Pino, R. E. (2013). Memristor SPICE model and crossbar simulation based on devices with nanosecond switching time. In IEEE international joint conference on neural networks (IJCNN).Google Scholar
  16. 16.
    Zidan, M. A., Fahmy, H. A. H., Hussain, M. M., & Salama, K. N. (2012). Memristor-based memory: The sneak paths problem and solutions. Microelectronics Journal, 44(2), 176–183.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of DaytonDaytonUSA

Personalised recommendations