Skip to main content
SpringerLink
Log in

Clock Gating-Based Effectual Realization of Stochastic Hyperbolic Tangent Function for Deep Neural Hardware Accelerators

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Comprehensive neural network applications led to the customization of a scheme to accelerate the computation on ASIC implementation. Hence, the determination of activation function in a neural network is an indispensable requisite. However, the specific design architecture of an activation function in a digital network encounters several difficulties as these activation functions demand additional hardware resources due to their non-linearity. This paper proposed an efficient hyperbolic tangent (tanh) function, wholly based on stochastic Computing methodology. The Hyperbolic tangent implementation is backed by the clock gating technique to curtail the dynamic power dissipation. The results are derived by implementing two different clock gating techniques on the proposed hardware. In this work, the proposed clock gating-based stochastic design for the implementation of activation function is efficient in terms of performance parameters such as area, power, and delay with negligible accuracy loss. MNIST dataset has been used for checking accuracy on LeNeT benchmark architecture. Furthermore, post-synthesis results show that the proposed clock gating design area is reduced by \(\approx \) 70.62\(\%\), power is reduced by \(\approx \) 58.19\(\%\), and delay is reduced by \(\approx \) 98.87\(\%\) compared to the state of the art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study, and detailed circuit simulation results are given in the manuscript.

References

  1. A.M. Abdelsalam, J.P. Langlois, F. Cheriet, A configurable FPGA implementation of the tanh function using DCT interpolation, in 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) (IEEE, 2017), pp. 168–171

  2. F. Albu, J. Kadlec, N. Coleman, A. Fagan, Pipelined implementations of the a priori error-feedback LSL algorithm using logarithmic arithmetic, in 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3 (IEEE, 2002), pp. III-2681

  3. F. Albu, J. Kadlec, N. Coleman, A. Fagan, The Gauss–Seidel fast affine projection algorithm, in IEEE Workshop on Signal Processing Systems (IEEE, 2002), pp. 109–114

  4. L. Benini, A. Bogliolo, G. De Micheli, A survey of design techniques for system-level dynamic power management. IEEE Trans. Very Large Scale Integr. Syst. 8(3), 299–316 (2000)

    Article  Google Scholar 

  5. L. Benini, G. DeMicheli, Dynamic Power Management: Design Techniques and CAD Tools (Springer, 1997)

    Google Scholar 

  6. M. Bhasin, G.P. Raghava, Prediction of CTL epitopes using QM, SVM and ANN techniques. Vaccine 22(23–24), 3195–3204 (2004)

    Article  Google Scholar 

  7. B.D. Brown, H.C. Card, Stochastic neural computation. I. Computational elements. IEEE Trans. Comput. 50(9), 891–905 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  8. C.H. Chang, H.Y. Kao, S.H. Huang, Hardware implementation for multiple activation functions, in 2019 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW) (IEEE, 2019), pp. 1–2

  9. G. Dinelli, G. Meoni, E. Rapuano, L. Fanucci, Advantages and limitations of fully on-chip CNN FPGA-based hardware accelerator, in 2020 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE, 2020), pp. 1–5

  10. M. Ercegovac, D. Kirovski, M. Potkonjak, Low-power behavioral synthesis optimization using multiple precision arithmetic, in Proceedings 1999 Design Automation Conference (Cat. No. 99CH36361), (IEEE, 1999), pp. 568–573

  11. B.R. Gaines, Stochastic computing systems. Adv. Inf. Syst. Sci. 2, 37–172 (1969)

    MATH  Google Scholar 

  12. S. Gomar, M. Mirhassani, M. Ahmadi, Precise digital implementations of hyperbolic tanh and sigmoid function, in 2016 50th Asilomar Conference on Signals, Systems and Computers, (IEEE, 2016), pp. 1586–1589

  13. K. Guo, L. Sui, J. Qiu, J. Yu, J. Wang, S. Yao, H. Yang, Angel-eye: a complete design flow for mapping CNN onto embedded FPGA. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 37(1), 35–47 (2017)

    Article  Google Scholar 

  14. H. Gyounghwan, Design Methodology for Cost Effective Clock and Power Gating (Doctoral dissertation, Seoul National University Graduate School, 2020)

  15. J. Kathuria, M. Ayoubkhan, A. Noor, A review of clock gating techniques. MIT Int. J. Electron. Commun. Eng. 1(2), 106–114 (2011)

    Google Scholar 

  16. K. Leboeuf, A.H. Namin, R. Muscedere, H. Wu, M. Ahmadi, High speed VLSI implementation of the hyperbolic tangent sigmoid function, in 2008 Third International Conference on Convergence and Hybrid Information Technology, Vol. 1 (IEEE, 2008), pp. 1070–1073

  17. B. Lee, N. Burgess, Some results on Taylor-series function approximation on FPGA, in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, vol. 2 (IEEE, 2003), pp. 2198–2202

  18. J. Li, Z. Yuan, Z. Li, C. Ding, A. Ren, Q. Qiu, J. Draper, Y. Wang, Hardware-driven nonlinear activation for stochastic computing based deep convolutional neural networks, in 2017 International Joint Conference on Neural Networks (IJCNN) (IEEE, 2017), pp. 1230–1236

  19. P. Li, D.J. Lilja, W. Qian, M.D. Riedel, K. Bazargan, Logical computation on stochastic bit streams with linear finite-state machines. IEEE Trans. Comput. 63(6), 1474–1486 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  20. C.W. Lin, J.S. Wang, A digital circuit design of hyperbolic tangent sigmoid function for neural networks, in 2008 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE, 2008), pp. 856–859

  21. J.Y.L. Low, C.C. Jong, A memory-efficient tables-and-additions method for accurate computation of elementary functions. IEEE Trans. Comput. 62(5), 858–872 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. D.T. Nguyen, T.N. Nguyen, H. Kim, H.J. Lee, A high-throughput and power-efficient FPGA implementation of YOLO CNN for object detection. IEEE Trans. Very Large Scale Integr. Syst. 27(8), 1861–1873 (2019)

    Article  Google Scholar 

  23. E. Nurvitadhi, J. Sim, D. Sheffield, A. Mishra, S. Krishnan, D. Marr, Accelerating recurrent neural networks in analytics servers: comparison of FPGA, CPU, GPU, and ASIC, in 2016 26th International Conference on Field Programmable Logic and Applications (FPL) (IEEE, 2016), pp. 1–4

  24. G. Rajput, G. Raut, M. Chandra, S.K. Vishvakarma, VLSI implementation of transcendental function hyperbolic tangent for deep neural network accelerators. Microprocess. Microsyst. 84, 104270 (2021)

    Article  Google Scholar 

  25. G. Rajput, S. Agrawal, G. Raut, S.K. Vishvakarma, An accurate and noninvasive skin cancer screening based on imaging technique. Int. J. Imaging Syst. Technol. 32(1), 354–368 (2022)

    Article  Google Scholar 

  26. G. Rajput, S. Agrawal, K. Biyani, S.K. Vishvakarma, Early breast cancer diagnosis using cogent activation function-based deep learning implementation on screened mammograms. Int. J. Imaging Syst. Technol. 32, 1101 (2022)

    Article  Google Scholar 

  27. G. Rajput, K.N. Biyani, V. Logashree, S.K. Vishvakarma, SCAN: streamlined composite activation function unit for deep neural accelerators. Circuits Syst. Signal Process. 41(6), 3465–3486 (2022)

    Article  Google Scholar 

  28. S.J.V. Rani, P. Kanagasabapathy, Multilayer perceptron neural network architecture using VHDL with combinational logic sigmoid function, in 2007 International Conference on Signal Processing, Communications and Networking, (IEEE, 2007), pp. 404–409

  29. F. Ratto, T. Fanni, L. Raffo, C. Sau, Mutual impact between clock gating and high level synthesis in reconfigurable hardware accelerators. Electronics 10(1), 73 (2021)

    Article  Google Scholar 

  30. G. Raut, S. Rai, S.K. Vishvakarma, A. Kumar, A CORDIC based configurable activation function for ANN applications, in 2020 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), (IEEE, 2020), pp. 78–83

  31. G. Raut, S. Rai, S.K. Vishvakarma, A. Kumar, RECON: resource-efficient CORDIC-based neuron architecture. IEEE Open J. Circuits Syst. 2, 170–181 (2021)

    Article  Google Scholar 

  32. B. Zamanlooy, M. Mirhassani, Efficient VLSI implementation of neural networks with hyperbolic tangent activation function. IEEE Trans. Very Large Scale Integr. Syst. 22(1), 39–48 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering, Indian Institute of Technology Indore, Khandwa Rd, Simrol, Madhya Pradesh, 453552, India

    Gunjan Rajput, V. Logashree, Kunika Naresh Biyani & Santosh Kumar Vishvakarma

Authors
  1. Gunjan Rajput
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Logashree
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kunika Naresh Biyani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Santosh Kumar Vishvakarma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santosh Kumar Vishvakarma.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rajput, G., Logashree, V., Biyani, K.N. et al. Clock Gating-Based Effectual Realization of Stochastic Hyperbolic Tangent Function for Deep Neural Hardware Accelerators. Circuits Syst Signal Process 42, 5978–6000 (2023). https://doi.org/10.1007/s00034-023-02373-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-023-02373-8

Keywords

Search

Navigation