1 Introduction

Witnessing the proliferation of the Internet of Things (IoT) technology in the past decades, more and more smart devices are integrated into our daily lives [1, 2]. As the dependence on IoT and modern communication technologies deepens in this digital age, IoT security has been recognized among the top concerns. Information assurance is an important aspect of the IoT design to ensure system integrity [3]. A lot of efforts have been reported to secure the IoT devices and networks, including access control [4, 5], data authentication [6], privacy preservation [7, 8], and security solutions against hardware-level attacks, i.e., side-channel attacks (SCA) [9, 10]. The wide spread of sophisticated cyberattack tools has also shed light on IoT security. As attackers find more innovative ways to break an IoT-based system, the need for more robust security mechanisms is becoming increasingly important.

Implementation of various cryptography algorithms is a challenging task for a security architect. The use of robust attack-proof cryptography algorithms does not guarantee that the system will be secure. If algorithms are not implemented properly in hardware, that could mean possible exposed back doors for an adversary to exploit. So, appropriate implementation of cryptographic algorithms in hardware is essential in ensuring that a system is attack-proof. Fortunately, the community has recognized the potential of specific security frameworks, like trusted computing (TC), to tie key platform operations to tamper-resistant onboard hardware modules (so-called TPMs) [11]. The cybersecurity team at Quarkslab recently discovered that TPMs are susceptible to malicious actors who could access the TPM interface and send malicious commands to the TPM [12]. Therefore, a TPM-enabled system is inadequate for hardening a system because the communication channel between the TPM and the hardware is not completely secured.

This paper proposes a Trusted HArdware-based Scalable Secure Architecture (THASSA) that can either build a security protection layer for existing IoT systems or serve as the framework for designing new secured IoT systems. THASSA allows the scalability of heterogeneous IoT devices; thus, it enables the expansion of the security framework to other subsystems. This is advantageous in a complex system comprising many subsystems. Additionally, it might be possible to use the secure path that THASSA established to offload cryptography operations or to communicate securely with other host processors or subsystems.

THASSA aims to address the vulnerability of information flow within IoT edge devices. In the prototype presented in this paper, standard encryption and hash techniques are implemented to facilitate the secure flow of critical information. The encryption and hash algorithms demonstrate a proof-of-concept of the secure architecture. However, the THASSA design is not constrained by a particular encryption or hash algorithm. Algorithms used for the THASSA experimental study here can be replaced with other robust cryptography algorithm(s) in future system-building efforts. In addition, the design could be tailored to meet specific security requirements for new systems.

THASSA hardens an IoT system by preventing an attacker from exploiting the system’s vulnerabilities. Hardware and software-based solutions are used to prevent an attack on the system. Various encryption techniques prevent man-in-the-middle (MITM) or replay attacks. The cryptography software library and trusted device are designed such that a rogue user cannot maliciously use the system and cause a security breach. The message formats used for message exchange by different entities are defined in such a way as to prevent exploitation of the digital signature from a hash function, thus preventing a birthday attack. Moreover, the secure system is designed to prevent hardware-based attacks. The hardware and software components are designed to prevent an attacker from learning the system’s workings, thereby minimizing the chances of successfully reverse-engineering the entire system.

The major contributions of this paper are summarized here:

  1. 1.

    We propose THASSA, a novel COTS-based configurable trusted hardware aiming at a security framework for IoT systems. The trusted hardware utilizes a COTS TPM device to store secret keys and execute cryptographic operations. This prevents the vulnerability of the IoT processor’s internal or main memory from revealing secret keys.

  2. 2.

    A dynamic key rolling scheme is introduced that provides mitigation against the attacker learning the secret keys used for encryption and decryption. The trusted device utilizing the TPM supports key rolling. The secret keys used for encryption and decryption between trusted hardware and an IoT edge device(s) are rolled after a few uses. As a result, key rolling will make manipulation difficult for an adversary.

  3. 3.

    The THASSA framework includes a robust message exchange mechanism that lets the trusted hardware connect automatically to other IoT edge devices in the same system or on different subsystems, as long as both the trusted hardware and the IoT edge device(s) are set up correctly. Thus, THASSA promotes modularity, interoperability, and scalability while ensuring the system’s integrity is intact.

The rest of the paper is structured as follows. Section 2 briefly reviews the state-of-the-art in this area and discusses related work. Section 3 explores the attack model. Section 4 presents the design objectives, system architecture, and operations of THASSA in detail. Section 5 reports the experimental study on top of a prototype and presents a comparison with multiple state-of-the-art schemes. Finally, Sect. 6 concludes this paper with some discussions.

2 State of the art and related work

In recent years, there have been fast developments in the field of IoT security, some notable ones being (1) improving the security of the IoT processing platform via software, (2) implementing security-enabled processors such as the ARM TrustZone, etc., (3) improving the extension mechanism of the chain of trust between IoT devices in a network via software, and (4) hardening the execution platform by utilizing hypervisors and virtual machines.

Using software to execute cryptographic operations is still the most popular form of security implementation found in IoT devices [13, 14]. The software could be easily ported to any IoT platform for securing user applications. Moreover, many lightweight cryptographic algorithms have been proposed for resource-constrained IoT devices. One of the drawbacks of using software-based and/or lightweight crypto algorithm-based security mechanisms is that they can be exploited if the execution platform is compromised. For example, Meltdown and Spectre’s attacks have shown that critical information can be leaked from the processor and be possibly exploited by an attacker [15,16,17]. From a cryptographic operations point of view, the secret key can be stolen if it is stored in clear format in non-encrypted RAM. As a result, cryptanalysis-resistant algorithms will still be vulnerable if the execution platform is designed poorly.

Blockchain technique is gaining a lot of interest [18], including smart public safety [19], digital IoT data market [20], and lightweight edge network security [21]. This could be used for IoTChain where trust is extended to all the devices in a system by authenticating each device [22]. Blockchain-based security promises lightweight cryptographic algorithm execution on resource-constrained IoT edge devices while maintaining the integrity of the system [23]. Other software-based Virtual Private Network (VPN) mechanisms are also being used to securely communicate with other IoT devices [24]. Although numerous studies are proving the efficacy of blockchain or VPN-based systems, the concept of rogue or malicious users is not adequately addressed in these studies. The secret key utilized for secure communication is often not rolled; as a result, anyone with knowledge of the working system can mount an attack [25].

Fig. 1
figure 1

Multiple subsystems

Full size image

Security-enabled processors are on the rise today. Most Apple Macbooks are equipped with security enclave processors that assist with authenticating the device and performing a secure boot [26, 27]. ARM has also introduced a secure enclave on some of its processors called the TrustZone [28, 29]. Intel processors now come with built-in cryptographic engines. Additionally, security co-processors like the Trusted Platform Module (TPM) have a hardware-based cryptographic engine that is resistant to side-channel attacks. These engines include a way to run secure boot on platforms that were not designed with security built-in [30]. One of the major drawbacks of security-enabled processors is the cost of manufacturing. These types of processors are manufactured on a small scale due to the nature of sensitive and proprietary design information leakage they might be exposed to. As a result, economies of scale cannot be achieved very easily. There is also always a question about the supply chain being compromised. Moreover, developing software in these types of processors is a very complicated process [31, 32]. Thus, security-enabled processors cannot be easily deployed in IoT systems yet.

The use of hypervisors is becoming a popular form of implementation today. Hypervisors have paved the way for virtualizing hardware components and sharing a physical hardware resource among multiple user processes [33]. Moreover, the virtualized hardware is considered to be more secure since the hardware registers are mapped to a virtualized memory map, which cannot be exploited by an attacker [34]. The virtual machines execute instructions in partitioned memory space, so the security vulnerability through interference by another application is greatly reduced [35]. However, what is often not addressed is the performance of virtualized hardware resources and their ability to serve multiple user processes. Reports are indicating that hypervisors can make a system very sluggish [36, 37].

3 IoT attack model

An IoT system is a collection of interconnected hardware units that operate independently towards a common goal, depending on the application of the system. There are four functional blocks in an IoT system: System Manager, Sensor(s), Actuator(s), and Monitor. The Monitor block collects data from the sensors. The data is provided to the system manager to make decisions regarding when and how to control the actuator. Figure 1 illustrates an example of multiple IoT subsystems. The system will reliably operate if all the functional blocks of the system or subsystems operate correctly. Attackers can exploit the system to cause undesirable effects. The following section will discuss some common attacks that can be carried out in a closed system. A system can encounter various forms of attack. Below, we list several types of well-recognized attacks on IoT systems and highlight the THSSSA design objectives that countermeasure these attacks. The detailed discussions of the design objectives are presented in Sect. 4.

  1. 1.

    Man-in-the-Middle attacks (MITM): In this type of eavesdropping attack, the attacker relays messages between two IoT edge devices. The attacker can sniff incoming and outgoing network packets from and to the sender and receiver, respectively. This could be catastrophic if the two parties are communicating security-critical information.

    THASSA Countermeasures: Design objectives 3, 6, and 9. Moreover, keys are rolled as part of the process.

  2. 2.

    Replay attacks: In this type of attack, the attacker sniffs incoming or outgoing network packets and can re-transmit or delay the transmission of such packets. Moreover, similar network packets could be constructed to study the effect on the system. This is a very good way to learn the cause and effect of a system and possibly reverse engineer the entire system. Artificial intelligence could facilitate this type of attack.

    THASSA Countermeasures: Design objectives 3, 6, and 9. Moreover, keys are rolled as part of the process.

  3. 3.

    Birthday attacks: In this type of attack, the attacker uses the brute force method to discover the original message from the digital signature. Since hash functions produce a fixed number of bytes for any message size, it is possible to obtain the same result of a hash function using different inputs, which could be exploited to determine the original message.

    THASSA Countermeasures: Design objectives 3, 6, and 9. Moreover, keys are rolled as part of the process.

  4. 4.

    Rogue user: In this type of attack, the attacker is already familiar with the system and is attempting to intentionally cause a system breakdown or utilize various replay attack methods to reverse engineer the system.

    THASSA Countermeasures: Design objectives 3, 4, and 8. The pre-determined encryption and hash function names are stored in an encrypted file in the trusted device. If, for some reason, the end user is knowledgeable about the algorithms in use, it will be challenging for the end user to compromise the system’s security. Because secret keys are often rolled and stored in hardware in the TPM and are not easily retrievable. If, for some reason, an attack does occur, the trusted device detects it and broadcasts an SOS message to dismantle the framework so further attack is prevented.

  5. 5.

    Hardware-based cyberattacks: In this attack, the attacker finds ways to hijack the hardware to perform a specific task and possibly reverse engineer to produce a counterfeit of the original hardware.

    THASSA Countermeasures: Design objectives 3, 4, 6, and 7. The trusted device utilizes a TPM that authenticates the device during boot-up. Since the trusted hardware is involved in every process, it will be challenging to communicate on the secure channel if a node is hijacked. The behavior anomaly of the hijacked node will eventually be detected, and the secure framework will be dismantled.

4 THASSA: design and methodology

In this paper, we propose a Trusted HArdware-based Scalable Secure Architecture (THASSA), which enables to building of a security protection layer for existing IoT systems or serves as the framework for designing new secured IoT systems. This section presented the design objectives, rationales, architecture, and major functionalities in detail.

4.1 THASSA design objectives

THASSA is an integration of hardware and software solutions from a system-level perspective to strengthen the weak points of a system that an attacker might exploit. The objectives for a secure system are stated below.

  • Objective 1: The secure architecture should be implementable atop an existing network-based system. This will allow systems that are not designed for security to easily implement the security measures that come with the proposed architecture.

  • Objective 2: The secure architecture should be implemented in a new system connected to a network. The secure architecture will inspire the design of a new network-based system with security priorities. The THASSA architecture can be the basis for the actual system architecture designed for a particular application.

  • Objective 3: The network-enabled hardware called the trusted device will be responsible for maintaining the security of the system. The trusted device shall be the root of the security framework in the secured system. Host processors are required to establish trust with the trusted device before being formally part of the secured system. The trusted device will deny all requests from host processors that are not part of its list of trusted host processors.

  • Objective 4: The trusted device should be able to inform the host processors should an attack take place. The trusted device shall be able to communicate to the host processors that a security breach has occurred. This will allow the host processors to lock their crypto libraries and prevent their further use when the system is known to be in a vulnerable state. Moreover, this will also prevent the attacker from reverse engineering the crypto library and discovering the operation mechanism of the secured system.

  • Objective 5: The trusted device could be used to execute cryptography operations offloaded from host processors. The trusted device shall be capable of executing cryptographic operations such as encryption or decryption of data supplied by a trusted host processor. The host processor will inform the trusted device of the encryption or decryption algorithm to use. The encrypted or decrypted data will be returned to the host processor via a network packet once the trusted device has completed running the crypto algorithm. Note that all communication between the host processor and the trusted device is encrypted, and the encryption or decryption method is unknown to the user.

  • Objective 6: The trusted device could be used to establish a secure data path between host processors in the same system. The trusted device shall be capable of establishing a secure data path over a network between two host processors, which could be used to transfer data or files. The host processors will be connected to the same network. The trusted device will encrypt or decrypt all of the data and files. Moreover, the information being transferred will be closely monitored for possible tampering.

  • Objective 7: The trusted device could be used to establish a secure data path between host processors in different sub-systems. The trusted device shall be capable of establishing a secure data path between host processors in different sub-systems. This will require a handshake between the trusted devices of the subsystems. The host processors will be connected to different subnets. All the data and files being transferred will be encrypted or decrypted by the trusted device in each system. The trusted device will closely monitor the transfer of information for potential tampering.

  • Objective 8: The trusted device will store an encrypted file containing device-specific information, including the type of encryption algorithm to use. The trusted device shall store a file containing the trust IDs and encryption algorithms for use by each host processor in the system. This file will serve as a lookup table for the trusted device every time a packet is sent or received to or from a host processor. The file will be encrypted to prevent tampering with it. The method of encryption used will be unknown to the user.

  • Objective 9: The trusted device and the host processor will exchange continuous messages. The payload of such messages will be arbitrary-sized random data. The trusted device and the host processors will continuously exchange messages when in operation. The messages will be of arbitrary length with a payload containing random values, which will be used for rolling keys for encryption and decryption operations. Messages will be exchanged alongside operation-related messages such as off-load cryptography and secure communication.

  • Objective 10: A cryptography library will be used in a host processor to perform any operation involving the trusted device. A cryptography Library shall be used in a host processor to facilitate the use of the secure system. The library will contain SW APIs that the end user could invoke to use off-load cryptography and secure communication features of the secure system. Note that this library will be locked once an SOS message is received from the trusted device to prevent its further use.

  • Objective 11: The host processor will support a lightweight encryption algorithm. The host processor shall use a lightweight encryption algorithm to communicate with the trusted device. There is little overhead for executing tiny encryption, thus, it is ideal for resource-constrained devices. The secret key used to encrypt/decrypt plain text is rolled at every transaction.

Table 1 summarizes the design and technical approaches that fulfill the objectives.

Table 1 THASSA technical approaches to fulfill the objectives
Full size table

4.2 Overview of a THASSA-secured IoT system

The proposed THASSA design consists of a trusted device, a network switch, and one or more IoT devices. All devices are connected to the network. Figure 2 is a conceptual illustration including only two IoT units, two Single Board Computers (SBC) like Raspberry Pi, SBC 1, and SBC 2, for simplicity. Any number of IoTs is allowed as long as they are supported by the network switch. The Random Number Generator (RNG) feeds the trusted device with encrypted random values for secret key generation, etc.

Fig. 2
figure 2

Secure system architecture

Full size image

The IoT host initiates a request for an off-load cryptographic operation to the trusted device. The cryptographic operations include the Tiny Encryption Algorithm (TEA), Advanced Encryption Standard (AES), and the Secure Hash Algorithm (SHA). The TEA is known for its low memory footprint and is less computation intensive due to the smaller number of source lines involved in implementing the algorithm, thus making it an ideal encryption algorithm with less overhead. The trusted device is designed to perform such operations optimally. This enables SBC 1 and SBC 2 to utilize their resources for normal processes, using less resources and power for cryptography operations. Moreover, cryptographic operations are performed securely on the trusted device, and the result(s) of the operation(s) are sent to the IoT hosts once the operations have been completed successfully. The trusted device could also be used for executing secure applications.

A secure data path can be formed between IoT 1 and IoT 2 using a trusted device. IoT 1 and IoT 2 do not have direct access to the secure path. Instead, the trusted device establishes a secure communication channel between each IoT and itself. It then relays the message from the sending IoT to the receiving IoT after it has inspected the message for possible tampering. This continuous inspection helps ensure the integrity of the secure data path. If a tamper is detected, then the trusted device halts all currently executing processes, tears down the secure channels, erases all sensitive data in its internal memory, and notifies the breach of the IoTs via an SOS message before shutting down. During normal operation, the IoT and trusted devices will continuously exchange messages. The messages will be of arbitrary size and contain random values. The packets with a random payload will pose a greater challenge to the attacker if he or she chooses to probe the Ethernet interfaces or wires or use port mirroring to sniff packets to recover sensitive data that is being transmitted or received over the Ethernet bus. The next few sections will describe the message format of the various messages involved in the architecture.

4.3 Off-load cryptography operation

When an application running on a host processor wants to offload a cryptography operation or send a secure message to another host processor, it initiates the process by sending a message containing an encrypted predefined request ID to the trusted device. The crypto application running on the trusted device decrypts the message once received. It then inspects the decrypted message for a valid packet header. Re-calculating the digital signature in the encrypted message is also part of the inspection. The trusted device uses the pre-determined (when establishing the secure framework) encryption/hash function to carry out the inspection. Suppose the packet header corresponds to the request-id for off-load crypto or secure message. In that case, the crypto application begins encrypting the plain text received from the host processor or triggers a series of events to establish a secure data path with another host, depending on the type of request.

Figure 3 shows the message format that the host uses to request an offload crypto operation from the trusted device. The request ID for each request to the trusted device is pre-defined. The data in the message format will contain encrypted plain text of arbitrary length. It will be encrypted using a lightweight encryption algorithm to reduce overhead from the host processor. To an attacker, the packet will appear to be holding all random values if sniffed using sophisticated tools. Moreover, the ongoing message exchange between the host and the trusted device will pose a further challenge to the attacker. The encryption methods used in the crypto library and the trusted device are unknown to the end-user. This further prevents any deliberate attempt at a data breach if the end-user goes rogue. The method of encryption jointly agreed upon by the host and the trusted device will be known to the end-user. So, even if the end-user identifies the exact packet containing the sensitive data, deciphering the encrypted data by reverse engineering the encryption or hash algorithm that is used will be very challenging.

Fig. 3
figure 3

Message format for off-load cryptography

Full size image
Fig. 4
figure 4

Source host initiating a request for secure communication

Full size image

Figure 4 illustrates the message exchange between the host and the trusted device to initiate a request for a secure data transfer. The API used for this is send_secure_msg, which will craft the message, encrypt it using AES, and finally send the packet to the network. This is the first step to establishing a secure connection with another host within or outside the isolated network.

Figure 5 depicts the message format used for requesting a secure connection to another host processor. The payload is AES-encrypted. The AES encryption will pose very low overhead on the host processor due to the limited size of the message. The trust-ID of the trusted device is not to be revealed and always remains unknown to the end user. It will be catastrophic if the trusted device is breached in any way. Thus, using AES will provide an extra layer of protection with less overhead.

Fig. 5
figure 5

Message format for secure communication

Full size image
Fig. 6
figure 6

Trusted HW sending request to host processor

Full size image

The trusted device will encrypt the plain text once the plain text is received from the host application. Figure 6 illustrates the interaction between the trusted device and the host processor when encryption has concluded. It then sends a cipher text message to the requesting host application.

Fig. 7
figure 7

Message format for encrypted data

Full size image

Figure 7 illustrates the message format for sending cipher text from the trusted device to the host processor. The request ID and trust ID fields are encrypted using a tiny encryption algorithm.

Fig. 8
figure 8

Encryption process algorithm

Full size image
Fig. 9
figure 9

Sending confirmation to source host processor

Full size image

The algorithm of the encryption process is depicted in Fig. 8. First, the data buffer is checked to ensure it is valid before handing off the data to begin encryption. Second, the type of encryption to perform is identified. The critical host-trusted device information is stored in an encrypted file, only accessible by the crypto application. This file will contain the type of encryption to be used for servicing off-load cryptography requests from host processors. Moreover, the file will also contain the trust IDs of all the host processors in the system, trusted devices, and host processors on other sub-systems. Finally, the output buffer is populated with the cipher text once the encryption has concluded successfully, or an error code is returned to the calling API when an error occurs.

4.4 Secure communication channel

Figure 9 illustrates a secure data channel’s interaction between the trusted device, source host, and destination host processors. Note that in a secure data channel between two host processors, the trusted device is the middleman, ensuring that the integrity of the data channel is not compromised. The trusted device sends a secure channel request to the destination host processor when a secure data channel request is received from the source host processor. The message to the destination is encrypted using a mutually agreed-upon encryption algorithm and hash function. The destination host will respond to the trusted device’s request. Upon receiving the response, the trusted device inspects the message to ensure it is from an authentic host. Again, the inspection will also involve re-calculating the hash function. Finally, the trusted device will send a message to the source host indicating if the secure channel request has been accepted.

Figure 10 depicts the message format used to send a secure channel request to the destination host from the trusted device. Figure 11 depicts the message format used to send a response from the trusted device to the source host.

Fig. 10
figure 10

Message format for secure channel request

Full size image
Fig. 11
figure 11

Message format for a response from a trusted device

Full size image
Fig. 12
figure 12

Encryption process algorithm

Full size image
Fig. 13
figure 13

Message format of secure communication payload

Full size image

Figure 12 illustrates the inspection algorithm used in the trusted device to validate packets for secure data transfer. The response message from the destination host is decrypted and the message header is validated to ensure that the correct expected message has been received. The Trust ID field is checked to determine if the user authorized a secure channel between the host and destination. If the secure channel is authorized, then the message header is replaced with the appropriate request ID and Trust ID to be sent to the source host. The message is then encrypted using the mutually agreed-upon encryption algorithm. Finally, the response message is sent off to the network for the host processor.

Figure 13 illustrates the message format used to transfer data from a source to a destination host over a secure channel. The request ID and Trust ID fields will be encrypted using the Tiny encryption algorithm. The payload will be encrypted using the mutually agreed-upon encryption algorithm between the trusted device and the source or destination processor.

4.5 Extending trust across subsystems

Figure 14 illustrates the different subsystems in a system. The trusted device maintains the security in each subsystem, which can be used for off-loading crypto processes and securely communicating with other host processes in the same system. Moreover, the trusted device can also extend security to another trusted device in a different subsystem for secure communication, as long as both systems are connected to a network.

Fig. 14
figure 14

Extending trust to other sub-systems

Full size image

The trusted device plays a key role in extending trust to other subsystems. One of the IoTs in a subsystem will initiate the request if it chooses to establish a secure data channel to another IoT in a different subsystem. Once the trusted device receives the request, the trust ID of the destination host is checked for validity. If the trust ID is valid, the trusted device sends a secure channel request message to the trusted device of the destination system. The trusted device in the destination system receives the message, validates it, and then responds to the trusted device of the requesting system. Then, the trusted device in the destination system sends a request message to the destination host. The host checks the trust ID field to determine if the request is from an authenticated host. After successful validation, the destination host responds to the trusted device. Then, the trusted device in the destination system relays the response to the trusted device in the source system. The trusted device in the source system inspects, which involes re-calculating the mutually-agreed upon hash function of the message and then sends an acceptance message to the source host when a secure path has been successfully established. Figure 15 portrays all the steps involved in extending trust.

Fig. 15
figure 15

Sequence diagram for trust extension

Full size image

4.6 Software APIs

The following software APIs are used to facilitate off-loading cryptographic operations and use the secure channel for data transfers. The end user will be able to use the APIs packaged in a cryptography library. The #ifdef in the cryptography library will be used to identify if the library is running on a trusted device or host processor and expose relevant platform-specific APIs for use.

  • send_crypto_msg: This API is used to send an encrypted message to the trusted device for off-load cryptography.

  • send_secure_msg: This API is used to send a request to the trusted device for establishing a secure path to another host processor.

  • send_crypto_rcvMsg: This API is used for receiving a message containing an encrypted payload from the trusted device.

  • send_secure_dstMsg: This API is used for requesting a secure path to a particular host processor in the system by a trusted device.

  • send_secure_trstMsg: This API is used to send an acknowledgment to the trusted device after accepting the request for the secure path by the host processor.

  • send_secure_source_msg: This API is used to send a confirmation message to the requesting host processor indicating that the requested secure path has been established.

4.7 Trusted device overview

The trusted device is the key component in maintaining the integrity of the secure system. It is also capable of tearing down the security framework when an attack is detected. In this section, we will explore how the hardware and software on a trusted device work in tandem to execute off-load cryptography operations and create a secure data path between host processors in the same or different sub-systems.

The CPU of the trusted device controls the cryptography engine. The cryptography engine executes the encryption and decryption of data provided by the processor. There is no direct data transfer between the processor and the cryptography engine; instead, the processor points the memory location of the buffers for plaintext or cipher text to the cryptography engine. The crypto engine then grabs the data off the RAM, performs encryption and decryption, and stores the result in a buffer in RAM. Moreover, the crypto engine uses a random number generator to encrypt plain text. The processor is also capable of performing encryption and decryption of data and files. Therefore, the overall cryptography operation workload can be balanced between the processor and the cryptography engine.

Fig. 16
figure 16

Software architecture of a trusted device

Full size image

Figure 16 illustrates the software architecture diagram of the trusted device. A boot loader in flash memory is the first piece of software that will execute after power is applied to the processor. Once the boot loader performs necessary diagnostic tests like POST, it handovers control to the operating system’s kernel. A board support package containing all the device drivers for the trusted device will be loaded. Built-in functions within the board support package will run autonomous diagnostic tests on the components external to the processor. Upon successful completion of those tests, the kernel will load the user space where multi-threaded security-critical applications will start executing. The Board Management, Secure Communication, Software Cryptography Engine, and Security applications will run as daemon processes. These applications will enable the trusted device to autonomously handle balancing workloads on cryptography resources, respond to host processor requests, perform software-based cryptography, and monitor the security framework of the system.

  • Board Management: This application will keep track of the number of cryptography operations in a queue and the operations currently executing in the Software and Hardware Cryptography Engines. After assessing the workload in each crypto engine, it will assign each task in the queue to either the hardware or software crypto engine for execution. This application stops running when a security breach occurs.

  • Secure Communication: This application will respond to host processor requests to establish a secure communication path between two host processors. Upon receiving a request, it will check the validity of the message, check the trust IDs of the host processors, and send messages to the destination host processor. A thread will start executing to monitor the newly established secure data path between the host processors. The application stops running when a security breach occurs.

  • Software Cryptography Engine: This application will execute cryptography operations on plain text or cipher text. A crypto software library is already in place to execute such operations when needed. Moreover, this application can perform cryptographic operations as multi-threaded operations. The application stops running when a security breach occurs.

  • Security: This application will monitor the integrity of the secured system. Random messages of arbitrary length will be exchanged between host processors and the trusted device. The application will take part in the message exchange with host processors. It is also responsible for tearing down the entire security framework when system security is compromised. Upon detection, it will trigger a signal to all daemon applications notifying them of the breach and send an SOS message to all the host processors. This application has access to all the buffers of the daemon applications. Finally, a shutdown signal is sent to the OS to disengage the trusted device from the system.

Upon receiving a request for off-load cryptography, a new thread is created in the software crypto engine. In the new thread, the API call to the on-board HW crypto engine is made. While the thread is listening for incoming data, the HW crypto engine transfers the plain text or cipher text from the RAM to local memory, performs cryptography, and returns the result to the thread. The result is packaged into a message and sent to the requesting host processor.

4.8 Hardware and software interactions

Figure 17 illustrates the secure communication flow between HW and SW. When a request for a secure data path is received from the host processor, the trusted device creates a new thread in the secure communication application. The message is validated, and the cipher text is decrypted to obtain the trust-id of the destination host processor. The trusted device then prepares a message, encrypts it using the HW crypto engine, sends it to the destination host processor, and waits for acknowledgment. Then, the thread runs in the background in monitor mode.

Fig. 17
figure 17

Secure communication HW and SW interaction

Full size image
Fig. 18
figure 18

Secure data transfer HW and SW interaction

Full size image

Figure 18 is the working flow for secure data transfer interaction between HW and SW. Once a request for secure data transfer has been received from the host processor, the trusted device validates the message, decrypts it, and validates the trust ID of the destination host processor. The data or file is then encrypted using the HW crypto engine, packaged into a message, and sent off to the destination host processor. The thread runs in monitor mode after servicing the secure data transfer request.

Fig. 19
figure 19

Trust extension HW and SW interaction

Full size image

Figure 19 presents the interaction between HW and SW in the case of trust extension. The trusted device receives the message for trust extension from a host processor. The message is validated and decrypted, and the host processor’s trust ID in another subsystem is validated. Message exchanges occur between the two trusted devices to establish a secure path between the two subsystems. Once the path has been established, the source trusted device sends a confirmation message to the requesting host processor, indicating that a secure path has been established. The thread will return to monitor mode once the trust has been extended.

5 Prototype implementation and experimental results

The proposed THASSA is a large, complex framework aiming at a comprehensive, holistic solution toward secure IoT networks. In this paper, we report a preliminary experimental study that explores its feasibility. A small-scale proof-of-concept prototype is built and tested for performance in cryptography offloading, secure communication, and trust extension.

5.1 Off-load cryptography

Figure 20 describes the system setup for demonstrating the off-load cryptography capability of the proposed architecture. The host processor 1 initiates an off-load cryptography request to trusted device 1. Upon receiving the request, the trusted device 1 performs an AES operation and sends the encrypted data to the host processor 1.

Fig. 20
figure 20

System setup for offload cryptography testing

Full size image

Figure 21 shows the working flow of the off-loading cryptography operation. The HW and SW interaction system consists of host processors, a trusted device, a random number generator, and a network switch. All the host processors and trusted devices are Raspberry Pi-based embedded systems. The prototype will demonstrate the implementation of the secure architecture in common embedded devices without deep diving into the various encryptions described in Sect. 4. The following will illustrate the hardware and software design used for implementing a secure system in the prototype.

Fig. 21
figure 21

Off-load cryptography HW and SW interaction

Full size image

The off-load cryptography is demonstrated by running binary files on host processor 1 and trusted device 1. The binary files are specific to the target they are executing on. The binaries are generated using an ARM GCC cross-compiler for the Raspberry Pi 4. Socket programming is used to utilize the TCP/IP stack of the operating systems. All the secure communications received occur at UDP port 21000 using only ETH0. The host processor sends a TEA-encrypted request message of 92 bytes to trusted device 1. It then receives 64 bytes of AES-encrypted data from trusted device 2. ‘ubuntu.21000’ is referred to as the host processor 1. The 92 bytes of data from host processor 1 to trusted device 1 are shown. Moreover, the encrypted data of 64 bytes from trusted device 1 can also be shown. Trusted device 1 received a message of 92 bytes. The AES-encrypted data of 64 bytes is sent to host processor 1.

The results of the timing in-between network packets are provided in Table 2. It took 557us for trusted device 1 to process the incoming request and send a packet with AES data to host processor 1.

Table 2 Off-load cryptography performance evaluation
Full size table

5.2 Secure communication

Figure 22 describes the system setup for demonstrating the secure communication capability of the proposed architecture. The host processor 1, 2, and trusted device 1 are all connected to the network. For secure communication between the host processors, host processor 1 initiates a request to the trusted device. The request is a UDP message containing the trust ID of host processor 2. Once the message is received by the trusted device, the UDP message is read to obtain the trust ID of the destination host processor. The trusted device then sends a request out to host processor 2 and waits for acknowledgment. The destination host processor receives the message and sends an acknowledgment to the trusted device. Once an acknowledgment has been received, the trusted device sends a confirmation to host processor 1, which concludes the establishment of a secure communication path between host processors 1 and 2. Every message exchange between host processors 1 and 2 will go through a trusted device. This ensures the integrity of the messages.

Fig. 22
figure 22

Experimental setup for secure communication test

Full size image

Secure communication is demonstrated by running binary files on host processors 1, 2, and the trusted device. The binary files are specific to the target they are executing on. The binaries are generated using an ARM GCC cross-compiler for the Raspberry Pi 4. Socket programming is used to utilize the TCP/IP stack of the operating system. All the secure communications received occur at UDP port 22000. The send-and-receive functionality is configured differently for each target. Ethernet communications between targets occur strictly using the eth0 interfaces of the Raspberry Pi 4 s. Eth0 is the only Ethernet interface in RPi 4 s. All the targets are connected to an unmanaged network switch. The WLAN is used to SSH into the targets. Host 1 sends a 30-byte message (‘Trust ID of Host Processor 2) to trusted device 1. It receives 41 bytes of confirmation messages from trusted device 1. Once the secure path has been established, it sends a secure message of 14 bytes.

Trusted device 1 received a 30-byte message (‘Trust ID of Host Processor 2) from host processor 1. It sent 29 bytes of a request message to host processor 2. It received 15 bytes of acknowledgment from host 2. Then it sent 41 bytes of confirmation messages to host 1. After the secure path has been established, host processor 1 then sends fourteen 14-byte secure messages to the trusted device. The trusted device then relayed the 14-byte secure messages to host processor 2. The host processor 2 received 29 bytes of a request message from trusted device 1. It then sent 15 bytes of acknowledgment to the trusted device. Once the secure path has been established, it receives 14 bytes of secure messages from trusted device 1, sent initially from host processor 1.

The results of the timing in-between network packets for secure communication are shown in Table 3. The secure path is established at 16 Bytes/us rate. The time taken to send a secure message from Host 1 to Host 2 is more than 56us/Byte because trusted device 1 inspects the incoming secure message for possible tampering. All the messages involved in establishing the secure path do not need inspection.

Table 3 Secure communication performance evaluation
Full size table

5.3 Trust extension capability

Figure 23 the system setup for demonstrating the trust extension capability of the THASSA. The host processor 1, 2, and trusted devices 1 and 2 are all connected to the network. Host Processor 1 initiates the request for trust extension. The request is a UDP message sent to trusted device 1. The trusted device 1 receives the message and sends out a request for trust extension to the trusted device 2. Upon receiving the request, trusted device 2 sends a request message to host processor 2. The host processor 2 receives the message and sends an acknowledgment to trusted device 2. Trusted device 2 relays the acknowledgment to trusted device 1. Finally, the trusted device 1 sends a confirmation to the host processor 2 that the secure path has been extended to the host processor 2. All the message exchanges between host processor 1 and host processor 2 go through trusted device 1 and trusted device 2 for message integrity checks.

Fig. 23
figure 23

Experimental setup for trust extension capability test

Full size image

The trust extension is demonstrated by running binary files on host processors 1, 2, and trusted devices 1 and 2, which are specific to the target they are executing on. The binaries are generated using an ARM GCC cross-compiler for the Raspberry Pi 4. Socket programming is used to utilize the TCP/IP stack of the operating system. All the trust extension messages are received on UDP port 23000 using eth0 interfaces for all targets. The send-and-receive functionality is configured differently for each target. All the targets are connected to an unmanaged network switch. The WLAN is used to SSH into the targets. The host processor 1 sends a request message of 29 bytes to trusted device 1. It then receives a confirmation message of 34 bytes from trusted device 1, indicating the trust has been extended to the host processor 2. Host processor 1 sends a 14 bytes secure message to host processor 2.

Trusted device 1 receives a request message of 29 bytes from host processor 2. Then a request message of 23 bytes is sent to trusted device 2. Trusted device 2 sends an acknowledgment of 13 bytes to trusted device 1. A confirmation message of 34 bytes is sent to host processor 1. 14 bytes of secure message is sent to the trusted device 2. Trusted device 2 receives a request of 23 bytes from trusted device 1. Then it sends a request of 29 bytes to host processor 2. Receives a message of 15 bytes from host 2. The 15 bytes of acknowledgment is sent to the trusted device 1. It sends the secure message of 14 bytes to host 2. Host 2 receives 29 bytes of request from trusted device 2. It then sends a 15-byte acknowledgment to trusted device 2. Finally, it receives a secure message of 14 bytes from host processor 1.

Table 4 describes the results for timing between network packets involved in establishing trust with host processor 2 in a different subsystem, and sending a secure message from Host 1 to Host 2. Similar to secure communication, the rate at which the secure path is established is faster than sending a secure message to the destination host processor. This is because both trusted device 1 and trusted device 2 inspect every incoming and outgoing message between the host processors.

Table 4 Trust extension capability performance evaluation
Full size table

5.4 Results summary

The summary of the prototype results is portrayed in Fig. 24. The results line up with the proposed NSSA design. Off-load cryptography takes the least time since it does not require a secure path for message exchanges between the trusted hardware and host processor. On the other hand, secure communication took relatively less time to establish a secure path than trust extension. This is because, in secure communication, all parties exist in the same domain under the same trusted hardware, whereas in trust extension, two or more trusted hardware are involved. Similarly, secure messages take more time to get across host processors in trust extension than secure communication because trust extension involves multiple trusted hardware devices inspecting every incoming and outgoing message, thus introducing latency. Overall, the results line up with the proposed NSSA design. However, it is important to note that the hardware used for the prototype is not optimized for cryptography operations. On crypto-optimized hardware, the results will be better.

Fig. 24
figure 24

Prototype test results

Full size image

5.5 Comparison with state-of-the-art methods

The THASSA method introduces a Trusted Hardware-based Scalable Secure Architecture for IoT systems, addressing various security concerns. In terms of performance, THASSA aims to strengthen system security by implementing a network-based security framework as an overlay network. The architecture integrates hardware and software solutions to fortify system weak points. The attack detection capabilities of THASSA are highlighted in its design objectives, particularly Objective 4, which empowers the trusted device to inform host processors of a security breach, enabling them to lock crypto libraries and prevent further use during a vulnerable state. This proactive response mechanism enhances the system’s ability to detect and mitigate attacks promptly. THASSA introduces novelty through secure communication channels, off-load cryptography operations, and extending trust across subsystems. The architecture’s ability to establish secure data paths, perform cryptographic operations offloaded from host processors, and seamlessly extend trust between subsystems showcases its innovative approach. However, weaknesses may arise if the trusted device is compromised or if attackers manage to exploit vulnerabilities in the hardware or software components. Regular monitoring, continuous exchange of arbitrary-sized random messages, and the ability to dismantle the security framework in case of an intrusion mitigate potential weaknesses and enhance the overall security posture. We consider the following features to describe different parameters, such as novelty, weakness, and performance of each system, as shown in Table 5. Table 6 shows the THASSA feature strengths over other security solutions.

Table 5 Comprehensive comparison with our method
Full size table
Table 6 THASSA strengths vs. other methods
Full size table

5.6 Discusisons and limitations

The prototype results in Tables 1, 2, 3, 4, 5 and 6 align with the proposed THASSA design. Off-load cryptography takes the least amount of time since it does not require a secure path for message exchanges between the trusted device and host processor. On the other hand, secure communication took relatively less time to establish a secure path compared to trust extension. This is because, in secure communication, all parties exist in the same domain under the same trusted device, whereas in trust extension, two or more trusted devices are involved. Similarly, it takes secure messages more time to get across host processors in trust extension compared to secure communication because trust extension involves multiple trusted devices inspecting every incoming and outgoing message.

There are some limitations to the proposed secure architecture. The entire security of the system could be dismantled if the trusted device is breached. Although compromising a trusted device is difficult, it is not impossible. If the trusted device is a procured COTS IoT, then the vulnerabilities of the hardware architecture could be exploited. Moreover, in a COTS IoT, additional software overhead will not ensure full system security. In other words, the trusted device will need to be custom-designed to ensure maximum HW and system security. This will incur additional costs. However, assuming the trusted device is custom-designed to ensure maximum security, the benefits of the proposed architecture would outweigh the costs.

Additionally, the security vulnerability of the host processor is overlooked. The expectation for implementing the proposed architecture is to utilize a host processor with decent security features. If a secure architecture is implemented in a system with poorly designed host processors with no regard for security, then parts of the system could be compromised. However, the entire system is not dismantled since the communication between the other host processors and the trusted device is still active for as long as security integrity is not violated or breached.

6 Conclusions

This paper proposes THASSA, a trusted hardware-based secure system architecture that is capable of performing off-load cryptography operations, secure communication, and extending trust to other systems via trusted devices. The cryptography operations are off-loaded to the trusted device. The off-load cryptography is network-based, so it can be implemented in an existing system or a newer system as opposed to having a dedicated interface between a host processor and a trusted device. This reduces the burden on host processors from having to execute complex encryption/decryption algorithms on the processor. Moreover, the trusted device adheres to a secure hardware architecture that guarantees the secure processing of data as opposed to a generic host processor.

A secure data path can be established between two or more host processors. The trusted device is involved in every message transaction. It continuously monitors incoming and outgoing messages to detect any message tampering or replay attacks by an attacker. The message formats are designed to hold encrypted data with a digital signature. The digital signatures are recalculated in the trusted device and matched with the digital signature in the message. This custom message format provides an added layer of challenge for the attacker to decipher the original message. The secure data path can be extended to another subsystem via the trusted device. Both the trusted devices will be involved in monitoring the integrity of the messages passed across, which makes it extremely challenging for an attacker to cause a breakdown of subsystems in an overarching system.