Skip to main content
SpringerLink
Log in

Artificial Neural Network Hardware Implementation: Recent Trends and Applications

  • Conference paper
  • First Online:
Computational Vision and Bio-Inspired Computing ( ICCVBIC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1108))

  • 2048 Accesses

Abstract

The human brain far exceeds contemporary supercomputers in tasks such as pattern recognition while consuming only a millionth of the power. To bridge this gap, recently proposed neural architectures such as Spiking Neural Network Architecture (SpiNNaker) use over a million ARM cores to mimic brain’s biological structure and behaviour. Such alternate computing architectures will exhibit massive parallelism and fault tolerance in addition to being energy efficient with respect to traditional Von-Neuman architecture. This leads to the emergence of neuromorphic hardware that can exceed performance in Artificial Intelligence (AI) driven remote monitoring embedded devices, Robotics, Biomedical devices and Imaging systems etc. with respect to traditional computing devices. In this review paper, we focus on the research conducted in the field of neuromorphic hardware development and the applications. This paper consists of the survey of various training algorithms utilized in hardware design of neuromorphic system. Further, we survey existing neuromorphic circuits and devices and finally discuss their applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hu, M., Li, H., Chen, Y., Wu, Q., Rose, G.S., Linderman, R.W.: Memristor crossbar-based neuromorphic computing system: a case study. IEEE Trans. Neural Netw. Learn. Syst. 25, 1864–1878 (2014)

    Article  Google Scholar 

  2. Liu, C., Hu, M., Strachan, J.P., Li, H.: Rescuing memristor-based neuromorphic design with high defects. In: IEEE Design Automation Conference, June 2017

    Google Scholar 

  3. Nafea, S.F., Dessouki, A.A., El-Rabaie, S., El-Sayed.: Memristor Overview up to 2015. Menoufia J. Electron. Eng. Res. (MJEER), 79–106 (2015)

    Article  Google Scholar 

  4. Camilleri, P., Giulioni, M., Dante, V., Badoni, D., Indiveri, G., Michaelis, B.: A neuromorphic a VLSI network chip with configurable plastic synapses. In: International Conference of HIS, September 2007

    Google Scholar 

  5. Jesus, O.D., Hagan, M.T.: Backpropagation algorithms for a broad class of dynamic networks. IEEE Trans. Neural Netw. 18, 14–27 (2017)

    Article  Google Scholar 

  6. Nuno-Maganda, M.A., Arias-Estrada, M., Torres-Huitzil, C., Girau, B.: Hardware ımplementation of spiking neural network classifiers based on backpropagation-based learning algorithms. In: IEEE International Joint Conference on Neural Networks, June 2009

    Google Scholar 

  7. Krestinskaya, O., Salama, K.N., James, A.P.: Learning in memristive neural network architectures using analog backpropagation circuits. IEEE Trans. Circuits Syst. I 66(2), 719–732 (2019)

    Article  Google Scholar 

  8. Aziz, N.A., Latif, M.A.K.A., Abdullah, W.F.H., Tahir, N.M., Zolkapli, M.: Hardware ımplementation of backpropagation algorithm based on CHEMFET sensor selectivity. In: IEEE International Conference on Control System, Computing and Engineering, January 2014

    Google Scholar 

  9. Krestinskaya, O., Salama, K.N., James, A.P.: Analog backpropagation learning circuits for memristive crossbar neural networks. In: IEEE International Symposium on Circuits and Systems (ISCAS), May 2018

    Google Scholar 

  10. Vo, H.M.: Implementing the on-chip backpropagation learning algorithm on FPGA architecture. In: IEEE International Conference on System Science and Engineering (ICSSE), July 2017

    Google Scholar 

  11. Vo, H.M.: Training on-chip hardware with two series memristor based backpropagation algorithm. In: IEEE International Conference on Communications and Electronics (ICCE), July 2018

    Google Scholar 

  12. Chakraborty, I., Roy, D., Roy, K.: Technology aware training in memristive neuromorphic systems for nonideal synaptic crossbars. IEEE Trans. Emerg. Top. Comput. Intell. 2(5), 335–344 (2018)

    Article  Google Scholar 

  13. Shouval, H.Z., Wang, S.S.-H., Wittenberg, G.M.: Spike timing dependent plasticity: a consequence of more fundamental learning rules. Front. Comput. Neurosci. 4, 19 (2010)

    Google Scholar 

  14. Song, S., Miller, K.D., Abbott, L.F.: Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3(9), 919–926 (2000)

    Article  Google Scholar 

  15. Markram, H., Gerstner, W., Sjostrom, P.J.: Spike-timing-dependent plasticity: a comprehensive overview. Front. Synaptic Neurosci. 4(2) (2012)

    Google Scholar 

  16. Jin, X., Rast, A., Galluppi, F., Davies, S., Furber, S.: Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware. In: IEEE World Congress on Computational Intelligence (WCCI), July 2010

    Google Scholar 

  17. Yousefzadeh, A., Masquelier, T., Serrano-Gotarredona, T., Linares-Barranco, B.: Hardware ımplementation of convolutional STDP for on-line visual feature learning. In: IEEE International Symposium on Circuits and Systems (ISCAS), May 2017

    Google Scholar 

  18. Pedroni, B.U., Sheik, S., Joshi, S., Detorakis, G., Paul, S., Augustine, C., Neftci, E., Cauwenberghs, G.: Forward table-based presynaptic event-triggered spike-timing-dependent plasticity. In: IEEE Biomedical Circuits and Systems Conference (BioCAS), October 2016

    Google Scholar 

  19. Frenkel, C., Indiveri, G., Legat, J.-D., Bol, D.: A fully-synthesized 20-gate digital spike-based synapse with embedded online learning. In: IEEE International Symposium on circuits and systems (ISCAS), May 2017

    Google Scholar 

  20. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., Amit, D.J.: Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. J. Neural Comput. 12(10), 2227–2258 (2000)

    Article  Google Scholar 

  21. Azghadi, M.R., Iannella, N., Al-Sarawi, S.F., Indiveri, G., Abbott, D.: Spike-based synaptic plasticity in silicon: design, implementation, application, and challenges. Proc. IEEE 102, 717–737 (2014)

    Article  Google Scholar 

  22. Covi, E., George, R., Frascaroli, J., Brivio, S., Mayr, C., Mostafa, H., Indiveri, G., Spiga, S.: Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons. J. Phys. D Appl. Phys. 51(34), 344003 (2018)

    Article  Google Scholar 

  23. Zhang, Y., Wang, X., Friedman, E.G.: Memristor-based circuit design for multilayer neural networks. IEEE Trans. Circuits Syst.–I 65(2), 677–686 (2018)

    Article  Google Scholar 

  24. Alom, M.Z., Taha, T.M., Yakopcic, C.: Memristor crossbar deep network ımplementation based on a convolutional neural network. In: International Joint Conference on Neural Networks (IJCNN), July 2016

    Google Scholar 

  25. Bayat, F.M., Prezioso, M., Chakrabarti, B., Nili, H., Kataeva, I., Strukov, D.: Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits. Nat. Commun. 9, 2331 (2018)

    Article  Google Scholar 

  26. Krishnaprasad, A., Choudhary, N., Das, S., Kalita, H., Dev, D., Ding, Y., Tetard, L., Chung, H.-S., Jung, Y., Roy, T.: Artificial neuron using vertical MoS2/Graphene threshold switching memristors. Sci. Rep. 9, 53 (2019)

    Article  Google Scholar 

  27. Rosenthal, E., Greshnikov, S., Soudry, D., Kvatinsky, S.: A fully analog memristor-based neural network with online gradient training. In: IEEE International Symposium on Circuits and Systems (ISCAS), May 2016

    Google Scholar 

  28. Choi, S., Jang, S., Moon, J.H., Kim, J.C., Jeong, H.Y., Jang, P., Lee, K.J., Wang, G.: A self-rectifying TaOy/nanoporous TaOx memristor synaptic array for learning and energy-efficient neuromorphic systems. NPG Asia Mater. 10, 1097–1106 (2018)

    Article  Google Scholar 

  29. Chen, Y., Li, H., Yan, B.: Challenges of memristor based neuromorphic computing system. Sci. China Inf. Sci. 61, 060425 (2018)

    Article  Google Scholar 

  30. Fukami, S., Borders, W.A., Kurenkov, A., Zhang, C., DuttaGupta, S., Ohno, H.: Use of analog spintronics device in performing neuro-morphic computing functions. In: IEEE Berkeley Symposium on Energy Efficient Electronic Systems and Steep Transistor Workshop (E3S), October 2017

    Google Scholar 

  31. Zhang, D., Zeng, L., Cao, K., Wang, M., Peng, S., Zhang, Y., Zhang, Y., Klein, J.-O., Wang, Y., Zhao, W.: All spin artificial neural networks based on compound spintronic synapse and neuron. IEEE Trans. Biomed. Circuits Syst. 10(4), 828–836 (2016)

    Article  Google Scholar 

  32. Fukami, S., Ohno, H.: Perspective: spintronic synapse for artificial neural network. J. Appl. Phys. 124(15), 151904 (2018)

    Article  Google Scholar 

  33. Sengupta, A., Parsa, M., Han, B., Roy, K.: Probabilistic deep spiking neural systems enabled by magnetic tunnel junction. IEEE Trans. Electron Device 63(7), 2963–2970 (2016)

    Article  Google Scholar 

  34. Torrejon, J., Riou, M., Araujo, F.A., Tsunegi, S., Khalsa, G., Querlioz, D., Bortolotti, P., Cros, V., Yakushiji, K., Fukushima, A., Kubota, H., Yuasa, S., Stiles, M.D., Grollier, J.: Neuromorphic computing with nanoscale spintronic oscillators. Nat. Lett. 547, 428–431 (2017)

    Article  Google Scholar 

  35. Grollier, J., Querlioz, D., Stiles, M.D.: Spintronic nano-devices for bio-inspired computing. Proc. IEEE 104, 2024–2039 (2016)

    Article  Google Scholar 

  36. Sengupta, A., Yogendra, K., Roy, K.: Spintronic devices for ultra-low power neuromorphic computation. In: IEEE International Symposium on Circuits and Systems (ISCAS), May 2016

    Google Scholar 

  37. Babu, V.S., Rose Katharine, A.A., Baiju, M.R.: Adaptive neuron activation function with FGMOS based operational transconductance amplifier. In: IEEE Computer Society Annual Symposium on VLSI, May 2016

    Google Scholar 

  38. Keles, F., Yildirim, T.: Low voltage low power neuron circuit design based on subthreshold FGMOS transistors and XOR implementation. In: International Workshop on Symbolic and Numerical Methods, Modeling and Applications to Circuit Design (SM2ACD) (2010)

    Google Scholar 

  39. Sridhar, R., Kim, S., Shin, Y.-C., Bogineni, N.C.: Programmable Analog Synapse and Neural Networks Incorporating Same, United States (1994)

    Google Scholar 

  40. Fernandez, D., Villar, G., Vidal, E., Alarcon, E., Cosp, J., Madrenas, J.: Mismatch-tolerant CMOS oscillator and excitatory synapse for bioinspired ımage segmentation. In: IEEE International Symposium of Circuits and Systems, May 2005

    Google Scholar 

  41. Flak, J., Laihot, M., Halonen, K.: Binary cellular neural/nonlinear network with programmable floating-gate neurons. In: IEEE International Workshop on Cellular Neural Networks and their Applications, May 2005

    Google Scholar 

  42. Lu, D.D., Liang, F.-X., Wang, Y.-C., Zeng, H.-K.: NVMLearn: a simulation platform for non-volatile-memory-based deep learning hardware. In: IEEE International Conference on Applied System Innovation (ICASI), May 2017

    Google Scholar 

  43. Nakada, K., Asai, T., Amemiya, Y.: Analog CMOS ımplementation of a CNN-based locomotion controller with floating-gate devices. IEEE Trans. Circuits Syst.–I 52(6), 1095–1103 (2005)

    Article  Google Scholar 

  44. Kele, F., Yldrm, T.: Pattern recognition using N-input neuron circuits based on floating gate MOS transistors. In: IEEE EUROCON, May 2009

    Google Scholar 

  45. Jin, Z. Cheng, A.C.: A self-healing autonomous neural network hardware for trustworthy biomedical systems. In: IEEE International Conference on Field Programmable Technology, December 2011

    Google Scholar 

  46. Rahnamaei, A., Pariz, N., Akbarimajd, A.: FPGA ımplementation of an ANN for detection of anthelmintics resistant nematodes in sheep flocks. In: IEEE Conference on Industrial Electronics and Applications (ICIEA), May 2009

    Google Scholar 

  47. Heller, S., Hugle, M., Nematollahi, I., Manzouri, F., Dumpelmann, M., Schulze-Bonhage, A., Boedecker, J., Woias, P.: Hardware implementation of a performance and energy-optimized convolutional neural network for seizure detection. In: IEEE Annual International Conference of IEEE Engineering in Medicine and Biology Society (EMBC), July 2018

    Google Scholar 

  48. Luo, J., Coapes, G., Mak, T., Yamazaki, T., Tin, C., Degenaar, P.: Real-time simulation of passage-of-time encoding in cerebellum using a scalable FPGA-based system. IEEE Trans. Biomed. Circuits Syst. 10(3), 742–753 (2016)

    Article  Google Scholar 

  49. Tan, E.T., Halim, Z.A.: Development of an artificial neural network system for sulphate-reducing bacteria detection by using model-based design technique. In: IEEE Asia Pacific Conference on Circuits and Systems, December 2012

    Google Scholar 

  50. Dabrowski, D., Jamro, E., Cioch, W.: Hardware ımplementation of artificial neural networks for vibroacoustic signals classification. Acta Physica Polonica Ser. A 118(1), 41–44 (2010)

    Article  Google Scholar 

  51. Menon, S., Fok, S., Neckar, A., Khatib, O., Boahen, K.: Controlling articulated robots in task-space with spiking silicon neurons. In: IEEE International Conference on Biomedical Robotics and Biomechatronics RAS/EMBS, August 2014

    Google Scholar 

  52. Liu, J., Harkin, J., McDaid, L., Halliday, D.M., Tyrrell, A.M., Timmis, J.: Self-repairing mobile robotic car using astrocyte neuron networks. In: IEEE International Joint Conference on Neural Networks (IJCNN) (2016)

    Google Scholar 

  53. Wang, D., Hao, Y., Zhu, X., Zhao, T., Wang, Y., Chen, Y., Chen, W., Zheng, X.: FPGA ımplementation of hardware processing modules as coprocessors in brain-machine ınterfaces. In: IEEE Annual International Conference of Engineering in Medicine and Biology Society (EMBS) (2011)

    Google Scholar 

  54. Kocaturk, M., Gulcur, H.O., Canbeyli, R.: Toward building hybrid biological/in silico neural networks for motor neuroprosthetic control. Front. Neurorobotics 9(8), 496 (2015)

    Google Scholar 

  55. Malekmohammadi, A., Mohammadzade, H., Chamanzar, A., Shabany, M., Ghojogh, B.: An efficient hardware implementation for a motor imagery brain computer interface system. Scientia Iranica 26, 72–94 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CMR Institute of Technology, Bengaluru, India

    Jagrati Gupta

  2. Ramaiah Institute of Technology, Bengaluru, India

    Deepali Koppad

Authors
  1. Jagrati Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deepali Koppad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jagrati Gupta .

Editor information

Editors and Affiliations

  1. Department of CSE, RVS Technical Campus, Coimbatore, India

    S. Smys

  2. Faculty of Engineering, Faculdade de Engenharia da Universidade do Porto, Porto, Portugal

    JoĂŁo Manuel R. S. Tavares

  3. Faculty of Engineering, Aurel Vlaicu University of Arad, Arad, Romania

    Valentina Emilia Balas

  4. School of Computing, Tokyo Institute of Technology, Tokyo, Japan

    Abdullah M. Iliyasu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gupta, J., Koppad, D. (2020). Artificial Neural Network Hardware Implementation: Recent Trends and Applications. In: Smys, S., Tavares, J., Balas, V., Iliyasu, A. (eds) Computational Vision and Bio-Inspired Computing. ICCVBIC 2019. Advances in Intelligent Systems and Computing, vol 1108. Springer, Cham. https://doi.org/10.1007/978-3-030-37218-7_40

Download citation

Publish with us

Policies and ethics

Search

Navigation