Abstract
This chapter shows that diverse algorithms with significantly complex data-flow can also be mapped to DIMA. The mapping of a convolutional neural network (CNN) and a sparse distributed memory (SDM) to DIMA is demonstrated. Algorithmic opportunities such as the use of error-aware training in a DIMA-based CNN and the use of ensemble decision-making in SDM can be exploited to compensate for non-ideal analog computations in DIMA leading to even greater energy savings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
P.N. Whatmough, S.K. Lee, H. Lee, S. Rama, D. Brooks, G.-Y. Wei, A 28nm SoC with a 1.2 GHz 568nJ/prediction sparse deep-neural-network engine with >0.1 timing error rate tolerance for IoT applications, in IEEE International Solid-State Circuits Conference (ISSCC) (2017), pp. 242–243
K. Mai, T. Paaske, N. Jayasena, R. Ho, W.J. Dally, M. Horowitz, Smart memories: a modular reconfigurable architecture, in ACM SIGARCH Computer Architecture News, vol. 28 (ACM, New York, 2000), pp. 161–171
D. Patterson, T. Anderson, N. Cardwell, R. Fromm, K. Keeton, C. Kozyrakis, R. Thomas, K. Yelick, Intelligent RAM (IRAM): chips that remember and compute, in IEEE International Solid-State Circuits Conference (ISSCC) (IEEE, Piscataway, 1997), pp. 224–225
M. Kang, M.-S. Keel, N.R. Shanbhag, S. Eilert, K. Curewitz, An energy-efficient VLSI architecture for pattern recognition via deep embedding of computation in SRAM, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 8326–8330
M. Kang, S.K. Gonugondla, A. Patil, N.R. Shanbhag, A multi-functional in-memory inference processor using a standard 6T SRAM array. IEEE J. Solid State Circuits 53(2), 642–655 (2018)
M. Kang, S.K. Gonugondla, M.-S. Keel, N.R. Shanbhag, An energy-efficient memory-based high-throughput VLSI architecture for convolutional networks, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015)
M. Kang, E.P. Kim, M.-S. Keel, N. R. Shanbhag, Energy-efficient and high throughput sparse distributed memory architecture, in IEEE International Symposium on Circuits and Systems (ISCAS) (2015), pp. 2505–2508
M. Kang, S. Gonugondla, A. Patil, N. Shanbhag, A 481pJ/decision 3.4M decision/s multifunctional deep in-memory inference processor using standard 6T SRAM array. arXiv:1610.07501 (preprint, 2016)
M. Yamaoka et al., A 300-MHz 25-μA/Mb-leakage on-chip SRAM module featuring process-variation immunity and low-leakage-active mode for mobile-phone application processor. IEEE J. Solid State Circuits 40(1), 186–194 (2005)
Y. LeCun, C. Cortes, MNIST handwritten digit database. AT&T Labs (2010). http://yann.lecun.com/exdb/mnist
T. Kobayashi, K. Nogami, T. Shirotori, Y. Fujimoto, A current-controlled latch sense amplifier and a static power-saving input buffer for low-power architecture. IEEE J. Solid State Circuits76(5), 863–867 (1993)
M. Kang, N.R. Shanbhag, In-memory computing architectures for sparse distributed memory. IEEE Trans. Biomed. Circuits Syst.10(4), 855–863 (2016)
Y. LeCun, L. Jackel, L. Bottou, A. Brunot, C. Cortes, J. Denker, H. Drucker, I. Guyon, U. Muller, E. Sackinger et al., Comparison of learning algorithms for handwritten digit recognition, in International Conference on Artificial Neural Networks, vol. 60 (1995), pp. 53–60
D. Strigl, K. Kofler, S. Podlipnig, Performance and scalability of GPU-based convolutional neural networks, in IEEE Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) (2010), pp. 317–324
C. Farabet, C. Poulet, J.Y. Han, Y. LeCun, CNP: an FPGA-based processor for convolutional networks, in IEEE International Conference on Field Programmable Logic and Applications (FPL) (2009), pp. 32–37
K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (preprint, 2014)
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778
W. Rieutort-Louis, T. Moy, Z. Wang, S. Wagner, J.C. Sturm, N. Verma, A large-area image sensing and detection system based on embedded thin-film classifiers. IEEE J. Solid State Circuits51(1), 281–290 (2016)
B. Moons, M. Verhelst, A 0.3-2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets, in IEEE Symposium on VLSI Circuits (VLSI Circuits) (2016), pp. 1–2
J.K. Kim, P. Knag, T. Chen, Z. Zhang, A 640M pixel/s 3.65 mW sparse event-driven neuromorphic object recognition processor with on-chip learning, in IEEE Symposium on VLSI Circuits (VLSI Circuits) (2015), pp. C50–C51
J.M. Cruz-Albrecht, M.W. Yung, N. Srinivasa, Energy-efficient neuron, synapse and STDP integrated circuits. IEEE Trans. Biomed. Circuits Syst.6(3), 246–256 (2012)
V. Garg, R. Shekhar, J.G. Harris, Spiking neuron computation with the time machine. IEEE Trans. Biomed. Circuits Syst.6(2), 142–155 (2012)
P. Kanerva, Sparse distributed memory (MIT Press, Cambridge, 1988)
P.J. Denning, Sparse distributed memory. Research Institute for Advanced Computer Science (NASA Ames Research Center, Mountain View, 1989)
E. Lehtonen, J.H. Poikonen, M. Laiho, P. Kanerva, Large-scale memristive associative memories. IEEE Trans. Very Large Scale Integr. Syst.22(3), 562–574 (2014)
M. Lindell et al., Configurable sparse distributed memory hardware implementation, in IEEE International Symposium on Circuits and Systems (ISCAS) (1991), 3078–3081
J. Saarinen et al., VLSI architectures of sparse distributed memory, in IEEE International Symposium on Circuits and Systems (ISCAS) (1991), pp. 3074–3077
J.D. Keeler et al., Notes on implementation of sparsely distributed memory, in NASA Research Institute for Advanced Computer Science (1986)
S.-I. Chien, I.-C. Kim, D.-Y. Kim, Iterative autoassociative memory models for image recalls and pattern classifications, in IEEE International Joint Conference on Neural Networks (IJCNN) (1991), pp. 30–35
I. Kim et al., High performance PRAM cell scalable to sub-20nm technology with below 4F 2 cell size, extendable to DRAM applications, in IEEE Symposium on VLSI Technology (VLSI Technology) (2010), pp. 203–204
S. Aritome, Advanced Flash memory technology and trends for file storage application, in IEEE International Electron Devices Meeting (IEDM) (2000), pp. 763–766
A. Verma, B. Razavi, Frequency-based measurement of mismatches between small capacitors, in IEEE Custom Integrated Circuits Conference (CICC) (2006), pp. 481–484
K. Kim, H. Mahmoodi, K. Roy, A low-power SRAM using bit-line charge-recycling technique, in International Symposium on Low Power Electronics and Design (ISLPED) (2007), pp. 177–182
S. Brink, S. Nease, P. Hasler, S. Ramakrishnan, R. Wunderlich, A. Basu, B. Degnan, A learning-enabled neuron array IC based upon transistor channel models of biological phenomena. IEEE Trans. Biomed. Circuits Syst.7(1), 71–81 (2013)
S. Ramakrishnan, R. Wunderlich, J. Hasler, S. George, Neuron array with plastic synapses and programmable dendrites. IEEE Trans. Biomed. Circuits Syst.7(5), 631–642 (2013)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Kang, M., Gonugondla, S., Shanbhag, N.R. (2020). Mapping Inference Algorithms to DIMA. In: Deep In-memory Architectures for Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-35971-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-35971-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35970-6
Online ISBN: 978-3-030-35971-3
eBook Packages: EngineeringEngineering (R0)