Abstract
The paper describes the implementation of systolic array-based hardware accelerator for multilayer perceptrons (MLP) on FPGA. Full precision hardware implementation of neural network increases resource utilization. Therefore, it is difficult to fit large neural networks on FPGA. Moreover, these implementations have high power consumption. Neural networks are implemented with numerous multiply and accumulate (MAC) units. The multipliers in these MAC units are expensive in terms of power. Algorithms have been proposed which quantize the weights and eliminate the need of multipliers in a neural network without compromising much on classification accuracy. The algorithms replace MAC units with simple accumulators. Quantized weights minimize the weight storage requirements. Quantizing inputs and constraining activations along with weights simplify the adder as well as further reduce the resource utilization. A systolic array-based architecture of neural network has been implemented on FPGA. The architecture has been modified according to Binary Connect and Ternary Connect algorithms which quantize the weights into two and three levels, respectively. The final variant of the architecture has been designed and implemented with quantized inputs, Ternary connect algorithm and activations constrained to +1 and −1. All the implementations have been verified with MNIST data set. Classification accuracy of hardware implementations has been found comparable with its software counterparts. The designed hardware accelerator has achieved reduction in flip-flop utilization by 7.5 times compared to the basic hardware implementation of neural network with high precision weights, inputs and normal MAC units. The power consumption also has got reduced by half and the delay of critical path decreased by three times. Thus, larger neural networks can be implemented on FPGA that can run at high frequencies with less power.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Courbariaux M, Bengio Y, David JP (2015) Binaryconnect: training deep neural networks with binary weights during propagations. In: Proceedings of advances in neural information processing systems, pp 3123–3131
Lin Z, Courbariaux M, Memisevic R, Bengio Y (2015) Neural networks with few multiplications. arXiv:1510.03009
Hubara I, Courbariaux M, Soudry D, El-Yaniv R, Bengio Y (2016) Binarized neural networks. In: Proceedings of advances in neural information processing systems, pp 4107–4115
Farabet C, Poulet C, Han JY, LeCun Y (2009) CNP: an FPGA-based processor for convolutional networks. In: International conference on field programmable logic and applications, 2009, FPL 2009, pp 32–37. IEEE
Farabet C, LeCun Y, Culurciello E (2012) NeuFlow: a runtime reconfigurable dataflow architecture for vision. In: Proceedings of snowbird learning workshop, Apr 2012
Gokhale V, Jin J, Dundar A, Martini B, Culurciello E (2014) A 240 GOps/s mobile coprocessor for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 682–687
Cavigelli L, Benini L (2017) Origami: a 803-GOp/s/w convolutional network accelerator. IEEE Trans Circuits Syst Video Technol 27(11):2461–2475
Andri R, Cavigelli L, Rossi D, Benini L (2016) YodaNN: an ultra-low power convolutional neural network accelerator based on binary weights. In: Proceedings of ISVLSI, pp 236–241
Murtagh P, Tsoi AC, Bergmann N (1993) Bit-serial systolic array implementation of a multilayer perceptron. IEE Proc E Comput Digit Tech 140(5):277–288
Gadea R, Cerdá J, Ballester F, Mocholí A (2000) Artificial neural network implementation on a single FPGA of a pipelined on-line backpropagation. In: Proceedings of the 13th international symposium on system synthesis, pp 225–230. IEEE Computer Society
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sreehari, R., Deepu, V., Arulalan, M.R. (2019). A Hardware Accelerator Based on Quantized Weights for Deep Neural Networks. In: Sridhar, V., Padma, M., Rao, K. (eds) Emerging Research in Electronics, Computer Science and Technology. Lecture Notes in Electrical Engineering, vol 545. Springer, Singapore. https://doi.org/10.1007/978-981-13-5802-9_93
Download citation
DOI: https://doi.org/10.1007/978-981-13-5802-9_93
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-5801-2
Online ISBN: 978-981-13-5802-9
eBook Packages: EngineeringEngineering (R0)