Skip to main content
SpringerLink
Log in

Design of Sparsity Optimized Photonic Deep Learning Accelerators

  • Chapter
  • First Online:
Embedded Machine Learning for Cyber-Physical, IoT, and Edge Computing

  • 331 Accesses

Abstract

Sparse neural networks can greatly facilitate the deployment of neural networks on resource-constrained platforms as they offer compact model sizes while retaining inference accuracy. Because of the sparsity in parameter matrices, sparse neural networks can, in principle, be exploited in accelerator architectures for improved energy efficiency and latency. However, to realize these improvements in practice, there is a need to explore sparsity-aware hardware-software co-design. In this chapter, we discuss a novel silicon photonics-based sparse deep neural network inference accelerator called SONIC. SONIC takes advantage of the high energy efficiency and low latency of photonic devices along with software co-optimization to accelerate sparse neural networks. Experimental analysis shows that SONIC can achieve up to 5.8× better performance per watt and 8.4× lower energy per bit than state-of-the-art sparse electronic neural network accelerators and up to 13.8× better performance per watt and 27.6× lower energy per bit than the best known photonic neural network accelerators, at the time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE. 86(11), 2278–2324 (1998)

    Google Scholar 

  2. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. (2014)

    Google Scholar 

  3. Park, J., Li, S., Wen, W., Tang, P.T.P., Li, H., Chen, Y., Dubey, P.: Faster CNNs with direct sparse convolutions and guided pruning. In: Proc. ICLR (2017)

    Google Scholar 

  4. Zhang, S., Du, Z., Zhang, L., Lan, H., Liu, S., Li, L., Guo, Q., Chen, T., Chen, Y.: Cambricon-X: an accelerator for sparse neural networks. In: MICRO (2016)

    Google Scholar 

  5. You, W., Wu, C.: RSNN: a software/hardware co-optimized framework for sparse convolutional neural networks on FPGAs. IEEE Access. 9 (2021)

    Google Scholar 

  6. Aimar, A., Mostafa, H., Calabrese, E., Rios-Navarro, A., Tapiador-Morales, R., Lungu, I.A., Milde, M.B., Corradi, F., Linares-Barrance, A., Liu, S.C., Delbruck, T.: NullHop: a flexible convolutional neural network accelerator based on sparse representations of feature maps. IEEE Trans. Neural Netw. Learn. Syst. 30(3), 644–656 (2019)

    Google Scholar 

  7. Waldrop, M.M.: The chips are down for Moore’s law. Nat. News. 530(7589) (2016)

    Google Scholar 

  8. Sunny, F., Mirza, A., Nikdast, M., Pasricha, S.: CrossLight: a cross-layer optimized silicon photonic neural network accelerator. In: DAC (2021)

    Google Scholar 

  9. Gu, J., Zhao, Z., Feng, C., Liu, M., Chen, R.T., Pan, D.Z.: Towards area-efficient optical neural networks: an FFT-based architecture. In: ASP-DAC (2020)

    Google Scholar 

  10. Liu, W., Liu, W., Ye, Y., Lou, Q., Xie, Y., Jiang, L.: HolyLight: a nanophotonic accelerator for deep learning in data centers. In: DATE, 2019

    Google Scholar 

  11. Dang, D., Chittamuru, S.V.R., Pasricha, S., Mahapatra, R., Sahoo, D.: BPLight-CNN: a photonics-based backpropagation accelerator for deep learning. ACM JETC. 17(4), 1–26 (2021)

    Google Scholar 

  12. Sunny, F., Mirza, A., Nikdast, M., Pasricha, S.: ROBIN: a robust optical binary neural network accelerator. In: ACM TECS (2021)

    Google Scholar 

  13. Sunny, F., Nikdast, M., Pasricha, S.: SONIC: a sparse neural network inference accelerator with silicon photonics for energy-efficient deep learning. In: Asia and South Pacific Design Automation Conference (ASP-DAC) (2022)

    Google Scholar 

  14. Sunny, F., Taheri, E., Nikdast, M., Pasricha, S.: A survey on silicon photonics for deep learning. ACM JETC. 17(4), 1–57 (2021)

    Google Scholar 

  15. Pasricha, S., Nikdast, M.: A survey of silicon photonics for energy efficient manycore computing. IEEE D&T. 37(4), 60–81 (2020)

    Google Scholar 

  16. Bahirat, S., Pasricha, S.: METEOR: hybrid photonic ring-mesh network-on-chip for multicore architectures. ACM Trans. Embedd. Comput. Syst. 13(3), 1–33 (2014)

    Google Scholar 

  17. Bahirat, S., Pasricha, S.: HELIX: design and synthesis of hybrid nanophotonic applica-tion-specific network-on-Chip architectures. IEEE international symposium on quali-ty electronic design (ISQED), 2014.

    Google Scholar 

  18. Bahirat, S., Pasricha, S.: 3D HELIX: design and synthesis of hybrid nanophotonic appli-cation-specific 3D network-on-chip architectures. Workshop on exploiting silicon photonics for energy efficient heterogeneous parallel architectures (SiPhotonics), 2014.

    Google Scholar 

  19. Bahirat, S., Pasricha, S.: A particle swarm optimization approach for synthesizing ap-plication-specific hybrid photonic networks-on-chip. IEEE international symposium on quality electronic design (ISQED), 2012.

    Google Scholar 

  20. Bahirat, S., Pasricha, S.: UC-PHOTON: a novel hybrid photonic network-on-chip for multiple use-case applications. IEEE international symposium on quality electronic design (ISQED), 2010.

    Google Scholar 

  21. Bahirat, S., Pasricha, S.: Exploring hybrid photonic networks-on-chip for emerging chip multiprocessors. IEEE/ACM international conference on hardware/software codesign and system synthesis (CODES+ISSS), 2009.

    Google Scholar 

  22. Chittamuru, S.V.R., Thakkar, I., Pasricha, S., Vatsavai, S.S., Bhat, V.: Exploiting process variations to secure photonic NoC architectures from snooping attacks. IEEE Trans. Comput. Aided Des. Integrat. Circuits Syst. 40(5), 850–863 (2021)

    Google Scholar 

  23. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: LIBRA: thermal and process varia-tion aware reliability management in photonic networks-on-chip. IEEE Trans. Multi-Scale Comput. Syst. 4(4), 758–772 (2018)

    Google Scholar 

  24. Chittamuru, S.V.R., Dharnidhar, D., Pasricha, S., Mahapatra, R.: BiGNoC: Acceler-ating big data computing with application-specific photonic network-on-chip archi-tectures. IEEE Trans. Parallel Distrib. Syst. 29(11), 2402–2415 (2018)

    Google Scholar 

  25. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: HYDRA: heterodyne crosstalk miti-gation with double microring resonators and data encoding for photonic NoC. IEEE Trans. Very Large Scale Integrat. Syst. 26(1), 168–181 (2018)

    Google Scholar 

  26. Chittamuru, S.V.R., Desai, S., Pasricha, S.: SWIFTNoC: a reconfigurable silicon-photonic network with multicast enabled channel sharing for multicore architec-tures. ACM J. Emerg. Technol. Comput. Syst. 13(4), 1–27 (2017)

    Google Scholar 

  27. Chittamuru, S.V.R., Pasricha, S.: Crosstalk mitigation for high-radix and low-diameter photonic NoC architectures. IEEE Des. Test. 32(3), 29–39 (2015)

    Google Scholar 

  28. Thakkar, I., Chittamuru, S.V.R., Pasricha, S.: Mitigating the energy impacts of VBTI aging in photonic networks-on-chip architectures with multilevel signaling. IEEE workshop on energy-efficient networks of computers (E2NC), 2018.

    Google Scholar 

  29. Pasricha, S., Chittamuru, S.V.R., Thakkar, I., Bhat, V.: Securing photonic NoC Ar-chitectures from hardware Trojans. IEEE/ACM international symposium on net-works-on-chip (NOCS), 2018.

    Google Scholar 

  30. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: SOTERIA: exploiting process varia-tions to enhance hardware security with photonic NoC architectures. IEEE/ACM de-sign automation conference (DAC), 2018.

    Google Scholar 

  31. Thakkar, I., Chittamuru, S.V.R., Pasricha, S.: Improving the reliability and energy-efficiency of high-bandwidth photonic NoC architectures with multilevel signaling. IEEE/ACM international symposium on networks-on-chip (NOCS), 2017.

    Google Scholar 

  32. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: Analyzing voltage bias and temper-ature induced aging effects in photonic interconnects for manycore computing. ACM system level interconnect prediction workshop (SLIP), 2017.

    Google Scholar 

  33. Dang, D., Chittamuru, S.V.R., Mahapatra, R.N., Pasricha, S.: Islands of heaters: a novel thermal management framework for photonic NoCs. IEEE/ACM Asia & South Pacific design automation conference (ASPDAC), 2017.

    Google Scholar 

  34. Thakkar, I., Chittamuru, S.V.R., Pasricha, S.: A comparative analysis of front-end and back-end compatible silicon photonic on-chip interconnects. ACM/IEEE system level interconnect prediction workshop (SLIP), 2016.

    Google Scholar 

  35. Thakkar, I., Chittamuru, S.V.R., Pasricha, S.: Run-time laser power management in photonic NoCs with on-chip semiconductor optical amplifiers. IEEE/ACM interna-tional symposium on networks-on-chip (NOCS), 2016.

    Google Scholar 

  36. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: PICO: mitigating heterodyne cross-talk due to process variations and intermodulation effects in photonic NoCs. IEEE/ACM design automation conference (DAC), 2016.

    Google Scholar 

  37. Chittamuru, S.V.R., Thakkar, I., Pasricha, S.: Process variation aware cross-talk mitigation for DWDM based photonic NoC architectures. IEEE international sym-posium on quality electronic design (ISQED), 2016.

    Google Scholar 

  38. Chittamuru, S.V.R., Pasricha, S.: SPECTRA: a framework for thermal reliability management in silicon-photonic networks-on-chip. IEEE international conference on VLSI design (VLSI), 2016.

    Google Scholar 

  39. Zhu, M.H., Gupta, S.: To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv:1710.01878v2, 2017.

    Google Scholar 

  40. Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv:1510.00149v5 [cs.CV], 2015.

    Google Scholar 

  41. Stefan, A., Stoferie, T., Marchiori, C., Caimi, D., Czornomaz, L., Stuckelberger, M., Sousa, M., Offrein, B.J., Fompeyrine, J.: A hybrid barium titanate–silicon photonics platform for ultraefficient electro-optic tuning. JLT. 34(8), 1688–1693 (2016)

    Google Scholar 

  42. Pintus, P., Hofbaurer, M., Manganelli, C.L., Fournier, M., Gundavarapu, S., Lemonnier, O., Gambini, F.: PWM‐driven thermally tunable silicon microring resonators: design, fabrication, and characterization. In: L&P (2019)

    Google Scholar 

  43. Xia, J., Bianco, A., Bonetto, E., Gaudino, R.: On the design of microring resonator devices for switching applications in flexible-grid networks. In: ICC, 2014

    Google Scholar 

  44. Lu, L., Li, X., Gao, W., Li, X., Zhou, L., Chen, J.: Silicon non-blocking 4×4 optical switch chip integrated with both thermal and electro-optic tuners. IEEE Photonics. 11(6) (2019)

    Google Scholar 

  45. Milanizadeh, M., Aguiar, D., Melloni, A., Morichetti, F.: Canceling thermal cross-talk effects in photonic integrated circuits. JLT. 37(4), 1325–1332 (2019)

    Google Scholar 

  46. Inti, R., Mansuri, M., Kennedy, J., Qiu, J., Hsu, C.M., Sharma, J., Li, H., Casper, B., Jaussi, J.: A scalable 32-to-56Gb/s 0.56-to-1.28pJ/b voltage-mode VCSEL-based optical transmitter in 28nm CMOS. In: CICC (2021)

    Google Scholar 

  47. Wang, B., Huang, Z., Sorin, W.V., Zeng, X., Liang, D., Fiorentino, M., Beausoleil, R.G.: A low-voltage Si-Ge avalanche photodiode for high-speed and energy efficient silicon photonic links. JLT. 38(12), 3156–3163 (2020)

    Google Scholar 

  48. Wu, B., Zhu, S., Xu, B., Chiu, Y.: A 24.7 mW 65 nm CMOS SAR assisted CT ΔΣ modulator with second-order noise coupling achieving 45 MHz bandwidth and 75.3 dB SNDR. IEEE J. Solid State Circuits. 51(12), 2893–2905 (2016)

    Google Scholar 

  49. Yang, C.M., Kuo, T.H.: A 3 mW 6-bit 4 GS/s subranging ADC with subrange-dependent embedded references. IEEE TCAS, 2021.

    Google Scholar 

  50. Shen, J., Shikata, A., Fernando, L.D., Guthrie, N., Chen, B., Maddox, M., Mascarenhas, N., Kapusta, R., Coln, M.C.W.: A 16-bit 16-MS/s SAR ADC with on-chip calibration in 55-nm CMOS. IEEE J. Solid State Circuits. 54(4), 1149–1160 (2018)

    Google Scholar 

  51. Zokaee, F., Lou, Q., Youngblood, N., Liu, W., Xie, Y., Jiang, L.: LightBulb: a photonic-nonvolatile-memory-based accelerator for binarized convolutional neural networks. In: DATE (2020)

    Google Scholar 

  52. Banerjee, S., Nikdast, M., Chakrabarty, K.: Modeling silicon-photonic neural networks under uncertainties. In: DATE 2021

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, 1373 Campus Delivery, Colorado State University, Fort Collins, CO, USA

    Febin Sunny, Mahdi Nikdast & Sudeep Pasricha

Authors
  1. Febin Sunny
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahdi Nikdast
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sudeep Pasricha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Febin Sunny .

Editor information

Editors and Affiliations

  1. Colorado State University, Fort Collins, CO, USA

    Sudeep Pasricha

  2. New York University Abu Dhabi, Abu Dhabi, Abu Dhabi, United Arab Emirates

    Muhammad Shafique

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sunny, F., Nikdast, M., Pasricha, S. (2024). Design of Sparsity Optimized Photonic Deep Learning Accelerators. In: Pasricha, S., Shafique, M. (eds) Embedded Machine Learning for Cyber-Physical, IoT, and Edge Computing. Springer, Cham. https://doi.org/10.1007/978-3-031-39932-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-39932-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-39931-2

  • Online ISBN: 978-3-031-39932-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation