Abstract
A Non-Volatile Memory (NVM) device-based deep inference architecture and a novel design of Swish activation function using analog components is proposed in this paper. The deep neural network is built using the 1T-1RRAM crossbar structure. This paper presents the importance of activation functions in analog hardware, compares the proposed Self-Gated activation function with the existing designs in literature, and implements a deep inference architecture using multiple datasets. The design has been evaluated for total power (peak), operating voltage, resistance characteristics, speed and the results indicate that the self-gated activation functions with RRAM device outperform Sigmoid & ReLU functions with memristors. The total power (peak) of the activation function circuit is reduced by 83.4% and the operating voltage by 60% compared to sigmoid with memristors and ON/OFF ratio by 23.49 compared to ReLU with memristors. The performance analysis of the inference architecture on iris, balance scale, and bank note authentication datasets have also been demonstrated. The observed classification accuracy of iris and bank note authentication datasets is 100% and 99.87% on the balance scale dataset. The analog hardware design of the deep neural network has been implemented in UMC 180 nm technology node, and the network has been trained offline using MathWorks®-MATLAB.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bersuker G, Mason M, Jones KL (2018) Neuromorphic computing: the potential for high performance processing in space. Center for space policy and strategy, pp 1–12. https://aerospace.org/sites/default/files/2018-12/Bersuker_NeuromorphicComputing_12132018.pdf
International Roadmap for Devices and Systems (2020) More Moore. IEEE. https://irds.ieee.org/editions/2020/more-moore
Chen PY, Yu S (2015) Compact modeling of RRAM devices and its applications in 1T1R and 1S1R array design. IEEE Trans Electron Devices 62(12):4022–4028
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, pp 198–201
Ramachandran P, Zoph B, Le QV (2017) Swish: a self-gated activation function. Google brain, pp.1–12, 16 Oct 2017
Hu M, Strachan JP, Li Z, Grafals EM, Davila N, Graves C, Lam S, Ge N, Williams RS, Yang J (2016) Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. Hewlett packard labs, 3 March 2016
Burr GW, Shelby RM, Sebastian A, Kim S, Kim S, Sidler S, Virwani K, Ishii M, Narayanan P, Fumarola A, Sanches LL, Boybat I, Le Gallo M, Moon K, Woo J, Hwang H, Leblebici Y (2017) Neuromorphic computing using non-volatile memory. Adv Phys: X 2(1):89–124
Kuzum D, Yu S, Wong H-SP (2013) Synaptic electronics: materials, devices and applications, IOP science. Nanotechnology 24(38)
Yu S, Sun X, Peng X, Huang S (2020) Compute-in-memory with emerging non-volatile-memories: challenges and prospects. IEEE custom integrated circuits conference (CICC). Boston, MA, USA, pp 1–4, 23 April 2020
Chou C et al (2018) An N40 256K×44 embedded RRAM macro with SL-precharge SA and low-voltage current limiter to improve read and write performance. In: IEEE international solid—state circuits conference—(ISSCC). San Francisco, CA, pp 478–480, 12 March 2018
Ye Z, Liu R, Barnaby H, Yu S (2019) Evaluation of single event effects in SRAM and RRAM based neuromorphic computing system for inference. IEEE international reliability physics symposium (IRPS). Monterey, CA, USA, pp 1–4, 23 May 2019.
Rajendran B, Alibart F (2016) Neuromorphic computing based on emerging memory technologies. IEEE J Emerg Sel Top Circuit Syst 6(2):198–211
Kaiyrbekov N, Krestinskaya O, James AP (2018) Variability analysis of memristor-based sigmoid function. In: International conference on computing and network communications (CoCoNet). Astana, Kazakhstan, pp 206–209, 19 May 2018
Bala A, Yang X, Adeyemo A, Jabir A (2019) A memristive activation circuit for deep learning neural networks. In: 8th International symposium on embedded computing and system design (ISED). Cochin, India, pp 1–5, 2 May 2019
Yu S (2018) Neuro-inspired computing With emerging nonvolatile memory. IEEE 106(2):260–285
Fisher RA (1988) UCI machine learning repository. Irvine. CA: University of California, School of Information and Computer Science, 1 July 1988. https://archive.ics.uci.edu/ml/datasets/iris
Siegler RS (1994) UCI machine learning repository. Irvine. CA: University of California, School of Information and Computer Science, 22 April 1994. http://archive.ics.uci.edu/ml/datasets/balance+scale
Lohweg V (2013) UCI machine learning repository. Irvine. CA: University of California, School of Information and Computer Science, 16 April 2013. https://archive.ics.uci.edu/ml/datasets/banknote+authentication
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Fatima, A., Pethe, A. (2021). NVM Device-Based Deep Inference Architecture Using Self-gated Activation Functions (Swish). In: Bajpai, M.K., Kumar Singh, K., Giakos, G. (eds) Machine Vision and Augmented Intelligence—Theory and Applications. Lecture Notes in Electrical Engineering, vol 796. Springer, Singapore. https://doi.org/10.1007/978-981-16-5078-9_4
Download citation
DOI: https://doi.org/10.1007/978-981-16-5078-9_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5077-2
Online ISBN: 978-981-16-5078-9
eBook Packages: Computer ScienceComputer Science (R0)