Abstract
The addition of FPGAs in the cloud is an emerging effort to support acceleration and performance with the flexibility of logic reprogramming. The underlying logic per unit area of the FPGA chip has multiplied, making it challenging for a single-user design to utilize completely and efficiently. Major service providers (such as Amazon, Alibaba, and Baidu) are moving toward a shared FPGA model that allows system designers to share the chip fabric either spatially or temporally. This virtual partitioning of FPGAs is comparable to the expeditionary systems that also adhere to the same principle of sharing chip fabric among multiple tenants. These tenants have the potential to execute any untrusted application on this shared hardware, which is a serious cause for concern in expeditionary systems. For instance, a tenant can deploy malicious circuits that compromise the confidentiality, integrity, and availability of its fellow tenants. In this paper, we investigate the threat landscape and propose mitigation strategies for multitenant FPGAs. We assess threats to the confidentiality of users’ critical data that are novel to the FPGA-as-a-Service (FaaS) framework. We present a defense mechanism for cloud FPGAs that verifies the integrity of tenants. In order to safeguard multi-tenant FPGAs from denial-of-service (DoS) attacks, our secondary defense mechanism promptly identifies malicious tenants and notifies the cloud orchestrator, thereby ensuring availability. We offer a comprehensive, all-in-one solution designed to defend and mitigate various threats faced by users in multi-tenant cloud FPGAs (in the public domain). The same principles apply to expeditionary systems with SWAP-constrained devices where multiple (potentially untrusted) applications share the same hardware. The proposed solution is thus adaptable and extendable to both public cloud service providers and expeditionary systems with private cloud infrastructure. The results show that the proposed work offers (i) safe-and-secure isolation of tenants, (ii) run-time access policy updates, and (iii) resilience against DoS attacks.
Similar content being viewed by others
Notes
Previous research has focused on either obtaining model hyperparameters or extracting secret cryptographic keys, remotely. In contrast, our work focuses on extracting model parameters remotely from an ML accelerator.
Vendors/CSPs may try to prevent this capability by detecting/restricting certain types of malicious circuits. However, these defenses can be bypassed by adversaries, as shown in Section 4.3.
We used Xilinx FD latch primitive to capture the output of buffer at each stage.
Functional verification of these efforts is beyond the scope of this work. This work does not claim these defenses as our own, but lists them for reader convenience.
AXI burst read/write operations.
AXI-Full interface is also supported, we prefer AXI-Lite for design simplicity.
TDC sensor can also be replicated multiple times for monitoring each tenant’s power individually. This allows for more fine-grained control, but at the cost of more area.
References
Xilinx (2022) Vivado design suite UG: dynamic function exchange (UG909). https://docs.amd.com/v/u/aKelve5HpAzsEgiAsRfyQA
Cloud A (2018) Deep dive into Alibaba cloud F3 FPGA as a service instances. Retrieved November 26:2020
Amazon (2019) Amazon AWS: Amazon EC2 F1 (2019) Amazon AWS. https://aws.amazon.com/ec2/instance-types/f1/
Xing J, Liu Y, Ge S, Li Y, Meng J (2019) An FPGA-based HF/VHF/UHF integrated self-interference cancellation system. 2019 Joint International Symposium on Electromagnetic Compatibility. Sapporo and Asia-Pacific International Symposium on Electromagnetic Compatibility (EMC Sapporo/APEMC), IEEE, pp 725–728
Module, Cobham AES Cryptographic Firmware-Hybrid (2015) Cobham TCS limited. https://seccerts.org/fips/f40ddf6742b7761b/target.pdf
Naghibijouybari H, Neupane A, Qian Z, Abu-Ghazaleh N (2018) Rendered insecure: Gpu side channel attacks are practical. Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. pp 2139–2153
Dubey A, Karabulut E, Awad A, Aysu A (2022) High-fidelity model extraction attacks via remote power monitors. 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, pp 328–331
Mahmoud DG, Lenders V, Stojilović M (2022) Electrical-level attacks on CPUs, FPGAs, and GPUs: survey and implications in the heterogeneous era. ACM Comput Surv (CSUR) 55(3):1–40
Karabulut E, Yuvarajappa C, Shaik MI, Potlurix S, Awad A, Aysu A (2022) Pr crisis: Analyzing and fixing partial reconfiguration in multi-tenant cloud fpgas. Proceedings of the 2022 Workshop on Attacks and Solutions in Hardware Security. pp 101–106
Restuccia F, Meza A, Kastner R (2021) AKER: A design and verification framework for safe and secure soc access control. 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, pp 1–9
Xilinx Inc (2022) PG059 AXI interconnect product guide. v2.1, PG059. https://docs.amd.com/r/en-US/pg059-axi-interconnect
Vipin K, Fahmy SA (2014) ZyCAP: efficient partial reconfiguration management on the Xilinx Zynq. IEEE Embed Syst Lett 6(3):41–44. IEEE
Sultana B, Ullah A, Malik AA, Zahir A, Reviriego P, Muslim FB, Ullah N, Ahmad W (2021) VR-ZYCAP: a versatile resourse-level ICAP controller for ZYNQ SOC. Electronics 10(8):99. MDPI
Vipin K, Fahmy SA (2012) A high speed open source controller for FPGA partial reconfiguration. 2012 International Conference on Field-Programmable Technology. IEEE, pp 61–66
Knodel O, Spallek RG (2015) RC3E: provision and management of reconfigurable hardware accelerators in a cloud environment. arXiv preprint http://arxiv.org/abs/1508.06843
Kao C (2005) Benefits of partial reconfiguration. Xcell Journal 55:65–67
Rihani MA, Nouvel F, Prévotet JC, Mroue M, Lorandel J, Mohanna Y (2016) Dynamic and partial reconfiguration power consumption runtime measurements analysis for ZYNQ SoC devices. 2016 International Symposium on Wireless Communication Systems (ISWCS). IEEE, pp 592–596
McDonald EJ (2008) Runtime FPGA partial reconfiguration. 2008 IEEE Aerospace Conference. IEEE, pp 1–7
Liu S, Pittman RN, Forin A, Gaudiot JL (2013) Achieving energy efficiency through runtime partial reconfiguration on reconfigurable systems. ACM T Embed Comput S (TECS) 12(3):593–660. ACM New York, NY, USA
Liu M, Kuehn W, Lu Z, Jantsch A (2009) Run-time partial reconfiguration speed investigation and architectural design space exploration. 2009 International Conference on Field Programmable Logic and Applications. IEEE, pp 498–502
Dubey A, Cammarota R, Aysu A (2020) BoMaNet: Boolean masking of an entire neural network. Proceedings of the 39th International Conference on Computer-Aided Design. pp 1–9
Moini S, Tian S, Holcomb D, Szefer J, Tessier R (2021) Remote power side-channel attacks on BNN accelerators in FPGAs. 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, pp 1639–1644
Karabulut E, Awad A, Aysu A (2023) SS-AXI: Secure and Safe Access Control Mechanism for Multi-Tenant Cloud FPGAs. 2023 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, pp 1–5
Xilinx User Guide 7 Series fpgas configuration. UG470, v1 11:1–176 11:1–176
Zynq X (2016) 7000 All Programmable SoC Overview. DS190 (v1. 10)0. (accessed on 2 Nov 2016)
AXI AAA, Specification-AXI AP (2011) Axi4, and axi4-lite, ace and ace-lite. tech rep, Technical report
Alam MM et al (2019) Ram-jam: remote temperature and voltage fault attack on FPGAs using memory collisions. In: Workshop on FDTC, IEEE, pp 48–55
Gnad DR, Oboril F, Tahoori MB (2017) Voltage drop-based fault attacks on FPGAs using valid bitstreams. 2017 27th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 1–7
Krautter J, Gnad DR, Tahoori MB (2018) FPGAHammer: remote voltage fault attacks on shared FPGAs, suitable for DFA on AES. IACR Trans Cryptogr Hardw Embed Syst 44–68
Zhao M, Suh GE (2018) FPGA-based remote power side-channel attacks. 2018 IEEE Symposium on Security and Privacy (SP). IEEE, pp 229–244
Dessouky G, Sadeghi AR, Zeitouni S (2021) SoK: Secure FPGA multi-tenancy in the cloud: Challenges and opportunities. 2021 IEEE European Symposium on Security and Privacy (EuroS &P. IEEE, pp 487–506
Dastres R, Soori M (2020) Impact of meltdown and spectre on cpu manufacture security issues. Int J Eng Sci Technol 18(2):62–69
Mittal S, Vetter JS, Li D (2014) A survey of architectural approaches for managing embedded dram and non-volatile on-chip caches. IEEE Trans Parallel Distrib Syst 26(6):1524–1537
Mittal S, Inukonda MS (2018) A survey of techniques for improving error-resilience of dram. J Syst Archit 91:11–40. Elsevier
Corbett JD (2012) The Xilinx isolation design flow for fault-tolerant systems. Xilinx White Paper WP412
Malik AA, Ullah A et al (2020) Isolation design flow effectiveness evaluation methodology for Zynq SoCs. Electronics 9(5):814
Krautter J, Gnad DR, Schellenberg F, Moradi A, Tahoori MB (2019) Active fences against voltage-based side channels in multi-tenant FPGAs. 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, pp 1–8
Pham K, Horta E, Koch D, Vaishnav A, Kuhn T (2018) IPRDF: An isolated partial reconfiguration design flow for Xilinx FPGAs. 2018 IEEE 12th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC). IEEE, pp 36–43
Giechaskiel I, Rasmussen KB, Eguro K (2018) Leaky wires: Information leakage and covert communication between FPGA long wires. Proceedings of the 2018 on Asia Conference on Computer and Communications Security. pp 15–27
Luo Y, Xu X (2019) HILL: A hardware isolation framework against information leakage on multi-tenant FPGA long-wires. 2019 International Conference on Field-Programmable Technology (ICFPT). IEEE, pp 331–334
Yazdanshenas S, Betz V (2018) Improving confidentiality in virtualized FPGAs. 2018 International Conference on Field-Programmable Technology (FPT). IEEE, pp 258–261
Yazdanshenas S, Betz V (2019) The costs of confidentiality in virtualized FPGAs. IEEE Trans Very Large Scale Integr (VLSI) Syst 27(10):2272–2283. IEEE
Hori Y, Satoh A, Sakane H, Toda K (2008) Bitstream encryption and authentication with AES-GCM in dynamically reconfigurable systems}. 2008 International Conference on Field Programmable Logic and Applications. IEEE, pp 23–28
Schellenberg F, Gnad DR, Moradi A, Tahoori MB (2021) An inside job: remote power analysis attacks on FPGAs. IEEE Des Test 38(3):58–66
Gravellier J, Dutertre JM, Teglia Y, Loubet-Moundi P (2019) High-speed ring oscillator based sensors for remote side-channel attacks on FPGAs. In: 2019 International conference on ReConFigurable computing and FPGAs (ReConFig), IEEE, pp 1–8
Glamočanin O, Coulon L, Regazzoni F, Stojilović M (2020) Are cloud FPGAs really vulnerable to power analysis attacks? 2020 Design, Automation \& Test in Europe Conference & Exhibition (DATE). IEEE, pp 1007–1010
Donchez S, Wang X (2022) Memory Isolation for Multi-Tenant Data Integrity in Cloud MPSoC FPGAs. 2022 IEEE 13th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). IEEE, pp 0515–0521
Versal ACAP (n. d.) Technical reference manual. Xilinx, uM011
Zynq UltraScale (n. d.) MPSoC Technical Reference Manual, UG1085
Tian S, Moini S, Wolnikowski A, Holcomb D, Tessier R, Szefer J (2021) Remote power attacks on the versatile tensor accelerator in multi-tenant FPGAs. 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp 242–246
Khawaja A, Landgraf J, Prakash R, Wei M, Schkufza E, Rossbach CJ (2018) Sharing, protection, and compatibility for reconfigurable fabric with AmorphOS. 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). pp 107–127
Korolija D, Roscoe T, Alonso G (2020) Do OS abstractions make sense on FPGAs? 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). pp 991–1010
Ma J, Zuo G, Loughlin K, Cheng X, Liu Y, Eneyew AM, Qi Z, Kasikci B (2020) A hypervisor for shared-memory FPGA platforms. Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. pp 827–844
Provelengios G, Holcomb D, Tessier R (2019) Characterizing power distribution attacks in multi-user FPGA environments. 2019 29th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 194–201
Boutros A, Hall M, Papernot N, Betz V (2020) Neighbors from hell: Voltage attacks against deep learning accelerators on multi-tenant FPGAs. 2020 International Conference on Field-Programmable Technology (ICFPT). IEEE, pp 103–111
Guide GS (2010) Logicore™ IP system monitor wizard v2. Citeseer
Nguyen M, Tamburo R, Narasimhan S, Hoe JC (2019) Quantifying the benefits of dynamic partial reconfiguration for embedded vision applications. 2019 29th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 129–135
Mahmoud D, Stojilović M (2019) Timing violation induced faults in multi-tenant FPGAs. 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, pp 1745–1750
Gravellier J, Dutertre JM, Teglia Y, Moundi PL, Olivier F (2020) Remote side-channel attacks on heterogeneous SoC. Smart Card Research and Advanced Applications: 18th International Conference, CARDIS 2019, Prague, Czech Republic, November 11--13, 2019, Revised Selected Papers 18. Springer, pp 109–125
Moini S (2023) Security of hardware accelerators in multi-tenant FPGA environments
Luo M, Suh GE (2021) Stealing zero-thresholding neural network data using timing channel
Hua W, Zhang Z, Suh GE (2022) Reverse-engineering CNN models using side-channel attacks. IEEE Des Test 39(4):15–22
Zhang Y, Yasaei R, Chen H, Li Z, Al Faruque MA (2021) Stealing neural network structure through remote FPGA side-channel analysis. IEEE Trans Inf Forensics Secur 16:4377–4388. IEEE
Güneysu T, Moradi A (2011) Generic side-channel countermeasures for reconfigurable devices. International Workshop on Cryptographic Hardware and Embedded Systems. Springer, pp 33–48
Danger JL, Guilley S, Bhasin S, Nassar M (2009) Overview of dual rail with precharge logic styles to thwart implementation-level attacks on hardware cryptoprocessors. 2009 3rd International Conference on Signals, Circuits and Systems (SCS). IEEE, pp 1–8
Wild A, Moradi A, Güneysu T (2017) GliFreD: Glitch-free duplication towards power-equalized circuits on FPGAs. IEEE Trans Comput 67(3):375–87. IEEE
Crane S, Homescu A, Brunthaler S, Larsen P, Franz M (2015) Thwarting cache side-channel attacks through dynamic software diversity. NDSS. pp 8–11
Luo Y, Xu X (2020) A quantitative defense framework against power attacks on multi-tenant FPGA. Proceedings of the 39th international conference on computer-aided design. pp 1–9
McEvoy RP, Murphy CC, Marnane WP, Tunstall M (2009) Isolated WDDL: A hiding countermeasure for differential power analysis on FPGAs. ACM Transactions on Reconfigurable Technology and Systems (TRETS) 2(1):1–23. ACM New York, NY, USA
Ma J, Li X, Wang M (2014) Power-aware hiding method for s-box protection. Electron Lett 50(22):1604–1606
Gnad DRE, Krautter J, Tahoori MB, Schellenberg F, Moradi A (2020) Remote electrical-level security threats to multi-tenant FPGAs. IEEE Des Test. https://doi.org/10.1109/MDAT.2020.2968248
Guilley S et al (2008) Evaluation of power-constant dual-rail logic as a protection of cryptographic applications in FPGAs. In: Second International Conference on Secure System Integration and Reliability Improvement, pp 16–23
Quisquater JJ (2002) Side channel attacks. CRYPTREC Report. https://cir.nii.ac.jp/crid/1572824500017724160
Mirzargar SS, Stojilović M (2019) Physical side-channel attacks and covert communication on FPGAs: A survey. 2019 29th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 202–210
Tiri K, Verbauwhede I (2004) Synthesis of secure FPGA implementations. Cryptology ePrint Archive
Yu P, Schaumont P (2007) Secure FPGA circuits using controlled placement and routing. Proceedings of the 5th IEEE/ACM international conference on Hardware/software codesign and system synthesis. pp 45–50
Moradi A, Standaert FX (2016) Moments-correlating DPA. Proceedings of the 2016 ACM Workshop on Theory of Implementation Security. pp 5–15
Knichel D, Moradi A, Müller N, Sasdrich P (2021) Automated generation of masked hardware. Cryptology ePrint Archive
Gao P, Xie H, Zhang J, Song F, Chen T (2019) Quantitative verification of masked arithmetic programs against side-channel attacks. International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, pp 155–173
Abromeit A, Bache F, Becker LA, Gourjon M, Güneysu T, Jorn S, Moradi A, Orlt M, Schellenberg F (2021) Automated masking of software implementations on industrial microcontrollers. 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, pp 1006–1011
Gnad D (2020) Remote attacks on FPGA hardware. Dissertation. Karlsruher Institut für Technologie (KIT), Karlsruhe
Benhani E, Bossuet L, Aubert A (2019) The security of arm trustzone in a FPGA-based SoC. IEEE Trans Comput 68(8):1238–1248
Factor M et al (2013) Secure logical isolation for multi-tenancy in cloud storage. 29th Symposium on Mass Storage Systems & Technologies. IEEE, pp 1–5
Hardy N (1988) The confused deputy: (or why capabilities might have been invented). ACM SIGOPS Operating Systems Review 22(4):36–38
Sun R, Qiu P, Lyu Y, Wang D, Dong J, Qu G (2021) Lightning: striking the secure isolation on GPU clouds with transient hardware faults. arXiv preprint http://arxiv.org/abs/2112.03662
La T, Pham K, Powell J, Koch D (2021) Denial-of-service on FPGA-based cloud infrastructures–attack and defense. IACR Transactions on Cryptographic Hardware and Embedded Systems. pp 441–464
La TM et al (2020) FPGADefender: malicious self-oscillator scanning for Xilinx UltraScale+ FPGAs. ACM Transactions on Reconfigurable Technology and Systems 13(3):1–31
Xilinx Inc (2021) PG267 AXI verification IP LogiCORE IP product guide. PG267
Farooq MO, Kunz T (2011) Operating systems for wireless sensor networks: a survey. Sensors 11(6):5900–5930
Hambarde P, Varma R, Jha S (2014) The survey of real time operating system: RTOS. 2014 International Conference on Electronic Systems. Signal Processing and Computing Technologies, IEEE, pp 34–39
Sabri C, Kriaa L, Azzouz SL (2017) Comparison of IoT constrained devices operating systems: A survey. 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA). IEEE, pp 369–375
Hong CH et al (2017) GPU virtualization and scheduling methods: a comprehensive survey. ACM Comput Surv(CSUR) 50(3):1–37
Yu F, Wang D, Shangguan L, Zhang M, Liu C, Chen X (2022) A survey of multi-tenant deep learning inference on GPU. arXiv preprint http://arxiv.org/abs/2203.09040
Salot P (2013) A survey of various scheduling algorithm in cloud computing environment. Int J Res Eng Technol 2(2):131–135
Chandiramani K, Verma R, Sivagami M (2019) A modified priority preemptive algorithm for CPU scheduling. Procedia Comput Sci 165:363–369. Elsevier
Reagen B, Adolf R, Shao YS, Wei GY, Brooks D (2014) MachSuite: benchmarks for accelerator design and customized architectures. In: IEEE International Symposium on Workload Characterization (ISWC), pp 110–119
Hara Y, Tomiyama H, Honda S, Takada H, Ishii K (2008) Chstone: A benchmark program suite for practical c-based high-level synthesis. 2008 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, pp 1192–1195
Zhou Y, Gupta U, Dai S, Zhao R, Srivastava N, Jin H, Featherston J, Lai YH, Liu G, Velasquez GA (2018) Rosetta: A realistic high-level synthesis benchmark suite for software programmable FPGAs. Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. pp 269–278
Gupta U, Dai S, Zhang Z (2015) Rosetta: A realistic benchmark suite for software programmable fpgas. Suite of Embedded Applications and Kernels Workshop (SEAK)
Goswami P, Shahshahani M, Bhatia D (2022) Mlsbench: A benchmark set for machine learning based fpga hls design flows. 2022 IEEE 13th Latin America Symposium on Circuits and System (LASCAS). IEEE, pp 1–4
Jamieson P, Rose J (2005) A verilog RTL synthesis tool for heterogeneous FPGAs. International Conference on Field Programmable Logic and Applications, 2005. IEEE, pp 305–310
Guo X, Dutta RG, He J, Tehranipoor MM, Jin Y (2019) Qif-verilog: Quantitative information-flow based hardware description languages for pre-silicon security assessment. 2019 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). IEEE, pp 91–100
Hansen MC et al (1999) Unveiling the ISCAS-85 benchmarks: a case study in reverse engineering. Des Test Comput 16(3):72–80
Das SR, Mukherjee S, Petriu EM, Assaf MH, Sahinoglu M, Jone WB (2006) An improved fault simulation approach based on verilog with application to ISCAS benchmark circuits. 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings. IEEE, pp 1902–1907
Elnaggar R, Karri R, Chakrabarty K (2019) Multi-tenant FPGA-based reconfigurable systems: attacks and defenses. 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, pp 7–12
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This work was funded through the Office of Naval Research (ONR) grant N00014-21-1-2809. The views, opinions, and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.
Conflict of Interest
The authors declare no competing interests. The work is done by NCSU and supported by ONR.
Author Contribution
All authors contributed to the design and implementation of the proposed evaluation methodology. All authors participated in the writing and revision of the paper. All authors have read and approved the submitted manuscript.
Availability of Data and Materials
These declarations are “not applicable.”
Ethical Approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Malik, A.A., Karabulut, E., Awad, A. et al. Enabling Secure and Efficient Sharing of Accelerators in Expeditionary Systems. J Hardw Syst Secur (2024). https://doi.org/10.1007/s41635-024-00148-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s41635-024-00148-4