1 Introduction

The Internet of Things (IoT) has experienced a swift surge in adoption, encompassing a diverse array of applications from personal health care and environmental monitoring to home automation, smart mobility, and Industry 4.0. Consequently, there has been a notable increase in the deployment of IoT devices in both public and private settings, becoming increasingly prevalent in households. With this widespread integration comes an escalated vulnerability to cybersecurity threats, necessitating measures to avert risks such as data breaches, denial-of-service attacks, and unauthorized network access. Addressing these challenges is crucial to ensure the secure and reliable operation of IoT systems across various applications.

The amount of recent assaults on embedded & IoT systems demonstrates their security risk. The Mirai botnet, for example, hijacked millions of IoT devices and coordinated them to conduct a distributed denial of service (DDoS) attack against several domain name system (DNS) servers, taking hundreds of thousands of websites offline throughout the world [1]. The Reaper malware, a more complex version of the Mirai, was originally disclosed in 2016 and specifically aimed at IoT devices having certain vulnerabilities rather than just credentials [2]. An Advanced Persistent Threat (APT) known as Black Energy caused a blackout by gaining supervisory control over various operating stations in 2014 [3]. Various other countries have also witnessed similar threats. Intruders obtaining control of more than 50 power plants, for example, could potentially compromise the electrical supply to 93 million Americans [4]. These real-world attacks show how IoT and embedded systems in key infrastructures can be severely harmed. Unfortunately, many commercial IoT goods do not often include sufficient security procedures, and as a result, they can be the target of or even the source of a variety of security threats. IoT and Embedded devices share various technical characteristics which include system architecture based on ARM or MIPS CPUs, Ethernet, Wi-Fi, or Bluetooth-based connectivity, and On-chip debugging interfaces such as UART, JTAG, I2C, or SPI. Most of these devices are controlled by vendor-specific software which is rarely updated to fix security problems. For a thorough security examination of these devices, proper identification of key technological aspects is critical. Furthermore, due to the diverse and non-standardized nature of the hardware and software features of embedded and IoT devices, security evaluation provides a number of issues. Security evaluation of IoT devices has two main aspects: network-based evaluation and firmware-based evaluation. In this work, we have mainly focused on the firmware part. In order to perform firmware security evaluation researchers have to get hold of the firmware and perform reverse engineering to reveal the vulnerabilities in it.

One of the most common causes of attacks on embedded systems has been identified as software vulnerabilities, and new vulnerabilities are discovered on a regular basis. Most of the popular binaries that are reused in software projects and firmware images are usually found vulnerable due to lack of security updates and due to this most of the embedded systems become implicitly vulnerable at an early stage. Several recent papers have also emphasised the importance of the analysis of firmware images [5]. Furthermore, in [6] Cui et al. claim that the third-party libraries used in firmware updates have been found to contain some of the famous vulnerabilities for years. They further reveal that about 80.4% of manufacturers distribute firmware with known flaws. As embedded systems manage critical components, compromising them could result in massive public system failures as well as serious security and safety implications, on a national or perhaps at a global scale. For example, 18 zero-day vulnerabilities were discovered in a Foscam IP camera, which includes insecure credentials, heap or stack buffer overflow, and command injection vulnerabilities [7].

Obsolete system architectures are also one of the main reasons for embedded systems being frequently vulnerable to attacks. Second, embedded systems’ internet connectivity, integration, and platform compatibility requirements make them more vulnerable to cyberattacks and exploitation. Finally, standard security techniques and traditional solutions, such as Intrusion detection or prevention systems cannot be used because these devices have limited computing power and memory. As a result, attackers take advantage of these flaws and create tailored malware for embedded systems and IoT devices.

Vulnerabilities in software can be found in both source and binary code. The latter techniques [8] use the source code to identify vulnerabilities. However, because most commercial software products are not open source, these techniques are not always viable. As a result, binary code analysis has become a necessity. Manual binary analysis, on the other hand, is a demanding, error-prone, and difficult process, particularly when dealing with a high number of embedded device firmware images. As a result, automated and scalable vulnerability identification is becoming increasingly important, in particular, it is highly desirable to scan a large number of firmware binaries for known and undiscovered vulnerabilities and produce a vulnerability analysis report in a timely manner. In this work, we have identified the architectural characteristics of IoT and embedded device firmware which include processor architecture, operating systems, bootloaders, protocols, and communication interfaces. Further, we discussed various firmware extraction techniques that are crucial in getting hold of IoT firmware. Furthermore, a detailed review of various vulnerability analysis frameworks is presented. A comparative analysis of these frameworks based on some common parameters is also provided. In the end, a new vulnerability analysis framework is proposed addressing some of the issues in already existing frameworks.

1.1 Research contributions

The following are the main contributions of this paper.

  • We have presented the architectural characteristics of IoT and embedded devices.

  • We have discussed the techniques for the extraction of firmware from IoT and embedded devices.

  • A comprehensive review of the state-of-art vulnerability analysis frameworks is presented with comparative analysis.

  • Finally, various challenges and gaps, facing in performing firmware analysis are given.

  • A new vulnerability analysis framework is proposed to address some of the challenges in the existing frameworks.

1.2 Methods and materials

We have used an advanced search approach to identify the related papers for our survey. We have mostly included papers from the most reputed journals of the IEEE, ACM, Wiley, Elsevier, and Springer publishers. Proper search strings with appropriate Boolean operators have been used in the advanced search such as "all in title: (“Firmware”) AND (“Vulnerability” OR “Analysis” OR “Security” OR “Blockchain” OR “Extraction”) source:” Springer” OR source:” ACM” OR source:” IEEE” OR source:” Wiley” OR source:” Elsevier”. We then filtered the results using various filters such as year of publication range and name of journals. The results were then properly filtered and various irrelevant papers were also removed.

1.3 Organisation

Table 1 presents the acronyms that are used in this paper. The rest of the paper is organized as follows. Section 2 is divided into three subparts—(A) Architecture of firmware and its various technical characteristics are discussed. (B) Various extraction methods are presented. (C) Various Firmware analysis frameworks are reviewed. Section 3 presents various challenges in firmware analysis. The proposed model is presented in Section 4. Conclusion and future work are given in Section 5.

Table 1 Acronyms used in the paper
Full size table

2 Background

This section focuses on the background of IoT-embedded firmware. This section is divided into four subsections. Firstly, we discuss IoT & Embedded Device Firmware Architecture. Secondly, we discuss the tools and techniques for the extraction of the firmware. In the third subsection, we focused on different types of vulnerability analysis methods. The fourth subsection presents various secure update mechanisms for the IoT device firmware.

2.1 Firmware architecture of IoT & embedded devices

The term “firmware” refers to binary software stored in an EEPROM or FLASH chip. The two available forms of firmware are low-level and high-level firmware. EEPROM usually stores the low-level firmware making it difficult to modify or update, while high-level firmware is stored in Flash memory. Firmware resides between the hardware and the application layer software, it works as an interface program for the software layer by realising the hardware commands. Firmware is the combination of various parts of binary files which include bootloader, OS kernel, file system, and various headers and because IoT devices have limited computational capabilities and storage space, firmware is frequently burnt in the compressed form [9]. IoT devices are more than just wireless sensors integrated into a gadget. The Internet of Things (IoT) is the connectivity of Wireless Sensor Network (WSN) devices with the Internet. The energy and memory resources available to IoT devices are typically limited. They’re usually tiny and battery-powered, having a memory capacity of around 100 kilobytes. Typical 8-bit microcontrollers are used in these machines, which are considerably behind the current generation of Windows/Unix/Mac-based PCs and laptops. Figure 1 presents the taxonomy of the commonly used IoT & embedded devices [10,11,12]. The devices have been classified under 4 broad categories viz Home Automation, Health/Fitness, Network/Routers, and Safety. These are further divided into 14 subcategories. The architectural characteristics of IoT and Embedded Device firmware have been presented in the subsequent section in terms of processor architecture, operating systems, bootloaders, kernel modules, and protocols.

Fig. 1
figure 1

Taxonomy of commonly used & most popular IoT devices [10,11,12, 28,29,30]

Full size image

2.1.1 Processor ISA

The architectures of embedded devices are quite varied. ARM and MIPS processors are widely used in the midrange to upper-class market sectors of processors that offer capabilities such as memory virtualization and high clock rates [13], and Intel is trying to catch up with its ATOM line. Processor designs with tiny memory and lower clock speeds, such as Atmel AVR or Intel 8051, are available in the lower-class market. The authors in [14] have analyzed approximately 9486 firmware images. The analysis resulted in identifying the various technical characteristics of the embedded device firmware including the identification of processor architectures, device operating systems, and protocols through the machine learning approach. The authors have reported that the majority of the firmware images which constitute around 79.4% of the analyzed firmware are based on MIPS 32 bit (Big Endian & Little Endian) architecture. The next most popular processor architecture is ARM 32bit (LE) which constitutes approximately 8.9% of the analyzed firmware. Another report given by Costin et al. [13] shows that after an automatic analysis of about 172,751 possible firmware images out of which 63% of them had ARM architecture and 7% were MIPS based. Together these constitute around 90% of the popular processor architectures used in IoT devices. The remaining portion consists of other different types of architectures. Figure 2 illustrates the architecture share among the analyzed firmware images presented by the authors in their research [14].

Fig. 2
figure 2

Types of firmware Instruction Set Architectures (ISA)

Full size image

Table 2 presents the study of various popular IoT devices. This also points to the fact that these devices are based on the popular ARM and MIPS-based architectures. The processors mainly used in these devices are based on ARM, TI, and AVR microcontrollers. These companies are the leading producer of semiconductors for IoT & Embedded devices.

Table 2 Processor and ISA specification of various identified IoT devices
Full size table

2.1.2 Operating systems

Firmwares of various levels of complexity power embedded systems. A full-fledged operating system, such as Linux or Windows NT, is generally used for more complicated ones. Operating systems like as VxWorks or Windows CE are used by less sophisticated devices, and a variety of special-purpose operating systems are also available. According to the findings given by authors in [14], around 40% of devices had Linux OS, 9% had Unix, and 3.5% had firmware based on VxWorks OS following signature analysis of device firmware. Other firmware had monolithic designs that didn’t have a particular kernel module. Figure 3 presents the most popular firmware OS used in IoT devices. Linux OS dominates the IoT landscape with a wide variety of library implementations/versions along with the same ABI-compatible Linux kernel with versions ranging (2.4 < x < 4.3) [31]. Table 3 presents the operating system specification of various identified IoT devices.

Fig. 3
figure 3

Popular operating systems used in IoT devices

Full size image
Table 3 Major operating systems of various IoT devices
Full size table

2.1.3 Bootloaders

The bootloader is the first programme that a system runs. It puts the kernel into memory for execution and initializes various hardware components which include Flash storage, I/O, RAM. The boot process in embedded systems can be divided into one, two, or three phases, with each step performing a particular function during startup. In a three-step procedure, the first bootloader conducts necessary hardware startup and loads the second-stage bootloader, which is usually located in ROM and is specific to microcontroller. The second stage bootloader initialises all the board-specific components and it usually resides on flash memory. After initialization, it loads the third stage bootloader, which loads the kernel into primary memory, initialises device drivers for the identified system components, and runs the kernel. U-Boot among the most popular second stage bootloaders for embedded devices [34]. A command-line interface is also available in it. Bootloader specification of some of the various widely used IoT devices is given in Table 4. It is evident from this table that the widely used bootloader is U-Boot.

Table 4 Various Bootloaders used in IoT devices
Full size table

2.1.4 Kernel modules

Kernel modules are small bits of code that may be loaded and unloaded from the kernel as needed. They improve the kernel’s functionality without requiring a system reset. Networking modules, cryptography modules, filesystem modules, and peripheral modules are some of the several types of kernel modules that may be found in embedded operating systems. The authors in [14] have analyzed that the networking modules consist of the largest share of the modules which approximates to 58.8% of the 504,815 identified firmware modules. The next largest part consists of peripheral related modules (12.6%) that include support for wireless adapters, chipsets, and I/O functionalities. Figure 4 shows the share of the kernel modules present in the analyzed IoT firmware.

Fig. 4
figure 4

Breakdown of types of kernel modules present in IoT devices

Full size image

2.1.5 Protocols

IoT devices employ different application layer protocols from the TCP/IP protocol stack, such as HTTP/HTTPS, FTP, Telnet, and ssh for authentication and data transfer. The authors in [14] have performed a network analysis of around 1971 firmware images and the analysis revealed that around 42% of the firmware supported the HTTP/HTTPS protocol. Remote shell access is supported by approximately 37% of the devices using the ssh or telnet protocol, however, 1.9% of the ssh supported devices also support telnet. Figure 5 presents a brief statistic of the commonly used TCP/IP stack protocols in the IoT devices.

Fig. 5
figure 5

Commonly used TCP/IP stack protocols in IoT devices

Full size image

2.2 Extraction techniques

When evaluating the safety of IoT devices, extracting the firmware is a critical first step. Preventing the firmware from any adversary is always desired from the standpoint of a designer: for example, to protect cryptographic keys used to recognize a device and prevent device cloning or intellectual property theft. Because of the wide range of IoT devices, multiple techniques for firmware extraction are required depending on the device.

Having access to the firmware of an embedded device may provide a lot of information about how it works and what vulnerabilities it has. Firmware frequently contains sensitive data like passwords and static keys, indicating unsafe design and poor security overall. Furthermore, the methods used for extraction of firmware provide the device write access, thus enabling the firmware to be modified. The extraction of firmware isn’t a precise science. IoT devices are very diverse due to their manufacturer specific configurations and software stacks. The firmware extraction of IoT devices is complicated by these specifications.

A number of methods for extracting firmware from a variety of IoT devices were given at DEFCON 25 [35], while authors in [36] emphasised on using the eMMC interface. The Exploitee.rs project was the result of this work. According to the research, The UART debugging interface has been found as the most exploitable programming & debugging interface in IoT devices. UART being vulnerable to firmware extraction in over 45 percent of the devices evaluated. Flash memory access is becoming increasingly crucial for contemporary gadgets. Notably, in virtually all situations when a hardware mechanism for firmware extraction is available, the approach also allows for firmware change and therefore device “rooting”. Extraction of firmware raises a number of difficulties for IoT device makers. For starters, there's a chance of losing intellectual property. More significantly, extraction of firmware can often lead to the discovery of security flaws in these devices. Due to severe vulnerabilities found, in some cases, this might have an impact not only on the examined device, but on all of the manufacturer’s products. The methods for extracting firmware are classified into three groups:

  1. 1.

    Utilising debug interfaces to get access to a local shell or read the contents of a memory.

  2. 2.

    Implementing a hardware memory dump on a flash chip.

  3. 3.

    Obtaining firmware access using software methods such as firmware upgrades and network eavesdropping.

2.2.1 Hardware methods

Firmware extraction through hardware methods uses the on-chip debug interfaces which include UART, JTAG, SPI, and I2C. The hardware method is often complicated in the process as it requires device-specific tools for carrying out the extraction. The three commonly used hardware-based methods for firmware extraction are discussed as follows.

2.2.1.1 Using UART

Direct access to an embedded device’s firmware through UART is usually a straightforward [37] method. Simply connecting to UART can lead directly to an unrestricted root shell. The Android debug interface ADB can sometimes provide access to a root shell on Android-based devices. Sometimes a root shell of a device is unavailable or is password secured then in that case a bootloader shell is used to get access to the firmware image. A internal dump of the complete filesystem is one of the main way to dump the firmware of a device with a live root access, all the archived files can then be unpacked using various open source tools. However, because embedded systems employ various types of flash storage with various filesystems, dumping block devices might cause issues. In general, the following steps are carried out when performing extraction of firmware through UART interface:

  1. 1.

    Visually inspect, oscilloscope probe, and trial-and-error to determine the UART interface;

  2. 2.

    An insecure shell can also be used to download the firmware image of a device. Netcat or related programmes and a computer on the same network can be used to download files.

  3. 3.

    If a shell is secured through password, guess all the default password pairing such as admin/admin. If shell is not accessible or no password is not accepted, try interrupting the boot process and entering the bootloader shell.

  4. 4.

    If you can't get into the bootloader shell, try momentarily disrupting the flash interface by grounding a data or clock pin or any other method while the bootloader loads the kernel.

2.2.1.2 Using JTAG

The JTAG connection that’s used to load firmware during production may well be used to read the chip’s entire memory. An appropriate programmer must be able to accept the memory dump and transfer it to a computer in order to read a device’s memory through a JTAG connection. After the gadget is manufactured, some manufacturers prevent it from being read or reprogrammed. The device is vulnerable to firmware extraction and injection attacks if the JTAG port is attached and unlocked. The typical procedure for extracting firmware through JTAG is as follows:

  1. 1.

    Manually identify JTAG or SWD debug port pins. JTAG offers variety of pin configuration ranging from 8 to 20 pins and SWD requires just two pins.

  2. 2.

    In using UART ground pin is identified first, when all the pins are identified a suitable UART debugger module is used to dump the contents of the internal memory.

  3. 3.

    Use datasheet to identify the pinout of specific microcontroller, JTAGulator [38] can also be used to identify the pins if datasheet is not available.

  4. 4.

    If no readout protection is activated, use an appropriate JTAG/SWD programmer to dump the internal memory.

Firmware extraction using JTAG becomes more complicated process due to the variety of pinouts and wide range of JTAG debuggers for different types of architectures. Thus, it is relatively easy to extract firmware through UART than JTAG.

2.2.1.3 Dumping flash

Directly accessing the flash storage is another method for hardware-based extraction. Older flash memory chips requires a lot of connections to the device and the use of specific programmer devices for transferring data efficiently. However, technologies like eMMC requires few connectors and can also be accessed with an SD card reader. Also, specific tools like easy RiffBox or JTAG Plus can also be used. A detailed process of extraction using eMMC can also be found in [39]. Some of the basic steps for performing flash dumps are as follows:

  1. 1.

    Determine the flash chip’s identity (based on a label, packaging type, and number of processor connections) and, if feasible, get a data sheet;

  2. 2.

    Use a datasheet or an oscilloscope to determine the pins. eMMC uses various pins which include CMD, CLK, and DAT0. CLK is a signal that repeats itself, whereas the CMD line includes brief data bursts that occurring before read or write of data on DAT0 pin.

  3. 3.

    Disable the processor’s access to eMMC first and link pins to an SD card, which may interface with using an SD card reader.

  4. 4.

    To access the contents of different flash chips, use an appropriate programmer, such as the MiniPro TL866;

  5. 5.

    If an in-circuit dump isn’t feasible, disassemble the flash chip and dump it using an appropriate reader.

It is necessary to restrict access from the board's CPU when accessing the memory for in-circuit dumps. This can be accomplished, for example, by momentarily disconnecting the clock line and reconnecting it once the dump is finished. Simply attaching an eMMC interface (such as easy JTAG Plus) might sometimes block the CPU from starting. Alternatively, you may use the appropriate pin to maintain the CPU in reset mode.

2.2.2 Software methods

Extraction of the firmware through software techniques does not require any access to the physical device. Examples include:

  1. 1.

    Browse for publicly accessible firmware on the device manufacturer's website.

  2. 2.

    Analyze the device’s network activity while following direct download URLs for firmware upgrades.

  3. 3.

    Use network traffic to intercept firmware upgrades. If TLS is in use, try to decrypt communications using self-signed certificates in a man-in-the-middle attack.

Vendors often provide firmware updates that only include revised files in that case complete firmware retrieval becomes difficult and alternate methods need to be explored. However, in certain situations, firmware upgrades contain entire firmware images, making this approach a quick and easy way to extract the firmware. Sometimes it is difficult to unpack some firmware images due to the implementation of firmware encryption or use of proprietary formats in compressing of firmware.

2.2.3 Firmware extraction tools

Firmware Extraction tools are broadly categorized into two classes: (1) Hardware tools, and (2) Software tools. The software tools are used in combination with supported hardware tools. The software tools can be freely downloaded online from their respective websites. Table 1 lists the various hardware tools that have been identified for the extraction of firmware using hardware-based methods. The various interfaces that are supported by these tools are UART, JTAG, SPI, I2C, and SWD. The cost of these tools typically ranges from $40 to $200 in the global market. These tools are typically used with their software counterpart which is usually free and open source. The tools include OpenOCD, Urtag, easy JTAG, and AVRdude, the details are listed in Tables 5 and 6.

Table 5 Various onboard programming & debugging tools
Full size table
Table 6 Supported software tools for firmware extraction
Full size table

2.2.4 Firmware unpacking & analysis tools

Firmware analysis is not quite straightforward and easy, and it necessitates a number of procedures prior to the analysis phase. Extraction, unpacking, and determining the file system, among other things, are all essential stages. After the firmware has been unpacked/extracted, it may be evaluated and analyzed for security. Using Binwalk [54], It is feasible to reverse engineer and do a rudimentary analysis on IoT device firmware images. Firmwalker [30] may be used to look for essential files such as private keys, certificates, and password files. IDA and Ghidra [55, 56] can be used to disassemble and debug even obfuscated code. For the emulation of the firmware, we can use QEMU [57]. Most of these tools are open-source which can be downloaded online. Table 7 presents all the tools which are used to do firmware-based evaluation & security profiling of IoT Devices.

Table 7 Various tools for firmware unpacking and analysis
Full size table

2.3 Vulnerability analysis frameworks

The security vulnerabilities can be found in various parts of an IoT system which include hardware components, application software [64, 65], underlying firmware, and cloud system [66,67,68,69,70]. Some of the various techniques used to find security flaws in IoT system are Static analysis, Dynamic analysis, Penetration testing, Fuzzing, and various other techniques.

Different sorts of vulnerabilities can be discovered with each approach. Identification of vulnerabilities in the embedded devices and in their underlying firmware serves a crucial role in securing embedded systems. To this end, there are a variety of methods for detecting and triggering possible vulnerabilities in deployed embedded system firmware. In this work, we give a detailed review of the some of the recent ideas, which utilize different analytic techniques, such as static, dynamic, and hybrid analysis approaches, to discover known and unknown firmware vulnerabilities.

2.3.1 Static analysis frameworks

Static analysis is used to find security flaws in firmware by analyzing the programme, which includes control, data flow, lexical, grammatical, and semantic analysis, among other things. Static analysis is a notion that has been around for a long time. It involves lexically examining a program's source code without running it [70]. Static analysis is used find various security flaws which include buffer overflows, type-checking errors, kernel deadlocks, susceptible function calls, and various other flaws. There are various analysis tools for such purposes which include Visual Code Grepper [71], CP-PCheck [72], PMD [73], Ghidra, IDA pro, and various other tools. Based on its implementation and targeted programming languages, each tool has its own technique of detecting mistakes. As IoT is made up of a variety of software components, APKs, and firmware, analyzing and detecting security flaws in these components is critical. In the past years, a substantial amount of study has been focused on firmware in general [74]. Static analysis techniques usually suffer from various limitations [75]. Although static analysis techniques are more scalable than dynamic analysis approaches, researchers are increasingly combining the two approaches as they both have their own set of constraints.

2.3.1.1 discovRE

Authors in [5] have developed and implemented a framework called discovRE, that supports four instruction set architectures which include × 86, × 64, ARM, and MIPS. It is a cross-architecture bug search framework for binaries. It works by matching a known vulnerable binary function with target firmware binaries typically compiled for different architectures, that contain the same vulnerable function. Two types are features are extracted prior to matching which are structural features and numerical features. Structural features are used to build a CFG (Control flow graph) of the binary. While numerical features represent the information about the number of instructions or number of basic blocks of a function. However, these CFG-based bug search approaches are far from being scalable to handle an enormous amount of IoT devices in the wild, due to their expensive graph matching overhead. This framework was evaluated on three firmware images and bugs like Poodle or Heartbleed were detected. Figure 6 shows the main process of this approach.

Fig. 6
figure 6

Architecture of discoverRE [5]

Full size image
2.3.1.2 Genius

A bug identification approach [76] that increases search accuracy while addressing the scalability challenge in existing tools like discoverRE. It constructs the attributed control flow graph using statistical and structural factors that are consistent across various CPU architectures and labels each basic block in a CFG with the set of attributes (ACFG). The ACFGs are transformed into codebooks using spectral clustering in order to do a more efficient search. However, it is stated by the authors that the creation of a codebook is computationally expensive.

2.3.1.3 BinArm

Authors in [77] presented a vulnerability detection technique called BinArm for smart grid IED firmware. It is a multistage detection engine that performs coarse to fine-grained detection as shown in Fig. 7. In the first stage, dissimilar functions having heterogeneous features are discarded. The second stage discards function based on different execution paths. The third stage identifies candidate functions using fuzzy graph matching based on weighted Jaccard similarity and Hungarian algorithm. It is proposed to be efficient in identifying vulnerabilities in IEDs in a smart grid system. However, the authors state that this system only performs analysis of ARM-based intelligent electronic devices and it fails to detect runtime exploits.

Fig. 7
figure 7

Proposed Binarm [77]

Full size image
2.3.1.4 FirmUp

This method is given by David et al. [78]. It identifies the vulnerable procedures by considering procedure-based relationships in firmware images. It establishes a correspondence between a set of procedures in a given binary and a target binary. An algorithm called Ehrenfeucht-Fraïssé [79] is used to establish a pairwise similarity between sets of procedures. This approach is tested on about 2000 firmware images and 373 vulnerabilities were discovered out of which 147 appeared in the latest firmware images.

2.3.1.5 XMATCH

This is a cross platform analysis framework given by Feng et all [80]. In this framework as shown in Fig. 8, three stage process is used for analysis. In the first stage, binary lifting is performed which produces an intermediate representation of the two binaries using usingMcSema [81] translator. In the next stage, conditional formulas are constructed from the lifted binaries. CF’s are used to capture two main factors of a bug, erroneous data dependency, and invalid conditional checks. Irrelevant variables are also discarded. In the third stage function matching is done using the already extracted conditional formulas and by employing integer programming techniques. After that one to one mapping is performed between the CF’s in addition to similarity scores.

Fig. 8
figure 8

XMatch approach overview [80]

Full size image
2.3.1.6 VulSeeker

Vulseeker is given by Gao et al. [82]. It is also a cross-platform approach based on function matching. The target function is compared with a vulnerable function and based on the similarity score the output is decided. Labelled semantic flow graph’s (LSFG) is constructed from the two binary functions then 8 types of instruction features are extracted as a numerical vector for each block of LSFG. After this, the numerical vector is fed into a DNN model to generate function semantics. The output is then decided based on the Cosine similarity score. The architecture is shown in Fig. 9.

Fig. 9
figure 9

Vulseeker Workflow [82]

Full size image
2.3.1.7 aDiff

This approach [83] extracts three types of features from binaries which are intra function, inter function, and inter-module features. The CNN and a Siamese network are used for the extraction of semantic features. After extraction of these features from two binaries, a distance measure is calculated between each pair of features of the two binaries. An overall similarity score is then obtained based on the three calculated distances.

2.3.2 Dynamic analysis frameworks

Dynamic analysis approaches rely on the firmware's real execution on hardware devices or emulators. By providing appropriate test inputs to analyze programme behavior, all of the firmware execution pathways are covered. In this part, we look at some of the most advanced dynamic analysis techniques for IoT and embedded device firmware. A comparative study is also supplied at the conclusion for a more in-depth comparison of the approaches mentioned.

2.3.2.1 Avatar

Jonas et al. have given a dynamic analysis framework for embedded devices called Avatar [13]. This framework shown in Fig. 10 works by a tight integration of an emulator with an embedded device for helping in various security tasks which include vulnerability discovery, vulnerability analysis, malware analysis, backdoor detection, and reverse engineering. An emulated firmware forwards I/O accesses to the real embedded devices thus completely emulating a full system behavior. Debug interfaces such as JTAG together with OpenOCD were used for communication with the real hardware device. The authors performed the analysis of three devices: a gsm-based phone, a hard disk bootloader, and a sensor node. Avatar supports all the major hardware architectures which include × 86–64, ARM, MIPS.

Fig. 10
figure 10

Avatar Architecture [13]

Full size image
2.3.2.2 Firmadyne

Chen et al. [84] have developed an automated dynamic vulnerability analysis system that supports full system emulation through QEMU. It specifically supports Linux-based devices. Firmadyne as shown in Fig. 11 consists of three major components which are Firmware Crawler for downloading firmware images from vendor websites, Firmware Extractor for extracting the downloaded file system, System Emulator for performing the initial emulation, and Dynamic Analyzer for running the exploits. Three types of architectures are supported by this framework which are MIPS-BE, MIPS-LE, and ARM-LE. The authors performed an extensive analysis in terms of the firmware count on about 9486 firmware. However, the dynamic analysis performed is rather simple in nature. The dynamic analyzer module consisted only of predefined exploits from the Metasploit framework and some custom-made exploits. These exploits are great in identifying the known vulnerabilities but are not effective in identifying zero-days.

Fig. 11
figure 11

Flow diagram of firmadyne [94]

Full size image
2.3.2.3 Automatic analysis framework

Costin et al. presented a dynamic analysis framework in [85]. In this framework, authors have used COTS tools for performing static and dynamic vulnerability analysis in the web interfaces of the embedded devices. Full-Scale emulation of 246 firmware images has been performed to test the web interfaces. The authors have found 225 high-impact vulnerabilities in around 24% of the emulated firmware. Tools such as RIPS, shodan, and ZMap were used for performing analysis. However, these tools have a limitation of producing high false negatives and false positives.

2.3.2.4 IoTFUZZER

IoT fuzzer is an automatic blackbox texting framework given by Chen et al. [86]. This framework aims at finding memory corruption vulnerabilities in firmware images by analyzing supporting apps. Dynamic analysis is performed on the app to reveal the logic that is used to construct the messages for communication with an IoT device. This framework has four main phases. In the first phase UI of the app is analyzed for the identification of components that trigger network connections. The second phase analyses the app for various strings and values which are required to construct a network-based protocol message. Then in the third stage, all the recorded protocol fields are used to construct a new message to be sent to the IoT device. The final stage monitors the status of the IoT devices and records any crashes or memory corruptions. The authors have evaluated this framework on 17 IoT devices and identified 15 memory corruption vulnerabilities. However, this framework provides only the input data that triggers the vulnerability and not the location of the vulnerability in the firmware.

2.3.2.5 Pretender

A dynamic analysis model called PRETENDER based on firmware re-hosting is given by Gustafson et al. [87]. In this model as shown in Fig. 12 interactions between firmware and hardware are recorded and then modelled using machine learning and pattern recognition techniques. After the completion of modeling, hardware is completely replaced with a virtualized environment. Virtualized environment is realized using QEMU [88] and for carrying out efficient program analysis angr is used. PRETENDER was evaluated on six firmware images of three different hardware types. PRETENDER was developed to provide an advanced approach for performing dynamic analysis on firmware images.

Fig. 12
figure 12

PRETENDER workflow diagram [87]

Full size image
2.3.2.6 FirmFuzz

Srivastava et al. [89] developed a dynamic vulnerability analysis framework of Linux-based IoT devices called FirmFuzz. It uses QEMU tool for carrying out emulation of the MIPS and ARM-based IoT firmware. There are three phases used in the analysis which are, Information gathering, Preparation, and Fuzzing. Firmware Fuzzing is the main technique used in identifying the vulnerabilities. It utilized the web interface of devices as entry points for fuzzing the firmware images. FirmFuzz managed to discover seven unknown vulnerabilities in six different devices by analyzing 32 images of 27 devices.

2.3.3 Hybrid analysis frameworks

The hybrid analysis is the combination of static analysis and dynamic analysis approaches. While designing hybrid analysis frameworks researchers frequently use various deep learning and machine learning methods to automate the process to a certain level. Very little work has been done in this area as a combination of both of the approaches presents some serious challenges. In this section, we have reviewed some of the existing work that has been done on hybrid approaches.

2.3.3.1 DTaint

It is a framework [90] to analyse taint style vulnerability in the embedded devices firmware. These types of vulnerabilities are weaknesses due to improper or no sanitization of input data. It has an input source, a specific data flow path, and a data sink that is sensitive in nature. A vulnerability such as the heartbleed [91] bug in the OpenSSL library is an example of a taint-style vulnerability. This framework as shown in Fig. 13 uses both static analysis and dynamic analysis techniques for the identification of vulnerability. In this framework firmware images are taken as input and outputs data flows from these images by using four components which are data structure, functional analysis, pointer aliasing and intraprocedural data flow components. The author applied this framework over 6 firmware images of four manufacturers and identified about 21 vulnerabilities including 13 zero days. However, this approach only identifies taint style vulnerabilities.

Fig. 13
figure 13

Dtaint Architecture [90]

Full size image
2.3.3.2 PATCHECKO

PATCHECKO is a state-of-the-art hybrid vulnerability analysis framework given by Sun et al. [92]. The architecture of PATCHECKO is shown in Fig. 14. It works in three phases: (1) It uses a deep learning technique to train the vulnerability detector. (2) Target firmware is of IoT/embedded devices is statically analyzed using the vulnerability detector. (3) The vulnerable functions identified during the second phase are dynamically analyzed to remove any false positives. Patchecko compares the functions with known CVE vulnerable functions and associated patches. Then vulnerable functions are produced as output with associated CVE numbers. Static analysis is used to convert each binary function into a feature vector. A deep learning-based model is used to compare two binary functions based on these feature vectors. After that, a more in-depth dynamic analysis is performed to remove any false positives. It has an accuracy of 93% for properly discovering known vulnerabilities, however, this framework does not identify any unknown vulnerabilities.

Fig. 14
figure 14

PATCHEKO workflow diagram [92]

Full size image

2.3.4 Comparative study

In this section, we compare the existing approaches for embedded systems as well as the traditional approaches that can potentially be applied to embedded systems. We further discuss our key observations from this comparative study. As per the comparative study of the frameworks, semantic and structural features based detection produces better results. The number of vulnerabilities produced by semantic and structural features is very high as compared to other techniques. Machine learning which includes deep learning-based approaches shows the best results for the detection of vulnerabilities in cross-architecture platforms. The frameworks are mainly evaluated on the major processor architectures which include x-86, MIPS, and ARM, which constitute the majority of the embedded and IoT devices. QEMU platform is mainly used in the dynamic analysis for runtime evaluation of the embedded firmware. Most of the static analysis tools employ function or pattern matching techniques to mainly detect known vulnerabilities and show poor performance in detecting unknown vulnerabilities. Among the static analysis frameworks, the FirmUp framework has detected a significant number of vulnerabilities across different architectures, whereas in dynamic analysis the framework given by Costin et all [85] has shown better results but across only two major architectures which include ARM, and MIPS. Hybrid analysis frameworks still need to improve in terms of vulnerability detection rate. Different machine learning methods may be explored for improving the hybrid analysis frameworks (Fig. 15).

Fig. 15
figure 15

Hierarchy diagram of vulnerability analysis frameworks

Full size image

The Table 8 provides an overview of various firmware analysis tools and their characteristics from 2014 to 2020. Several trends can be observed from the data. Over the years, there has been a shift from static analysis to dynamic analysis, including full system emulation and machine learning-based approaches. The number of supported architectures has also increased, accommodating a wide range of devices. Additionally, the number of firmware/devices analyzed has grown significantly, indicating the expanding scope of firmware analysis. Tools like “IoT FUZZER” and “FirmUp” have been designed for dynamic analysis, while “BinArm” and “VulSeeker” focus on static analysis. The development of machine learning and deep learning techniques is evident in tools like “Automatic Analysis,” “Genius,” and “PATCHECKO.” This comprehensive analysis landscape showcases the growing importance of firmware analysis in addressing cybersecurity challenges in the IoT and embedded device domain.

Table 8 Comparative analysis of the discussed vulnerability analysis frameworks
Full size table

2.3.5 Vulnerability proritization

Vulnerability prioritization plays a pivotal role in crafting an effective cybersecurity strategy, as it empowers organizations to allocate their resources judiciously while addressing the most imminent threats. This process entails a comprehensive evaluation of vulnerabilities, taking into account both their potential impact and exploitability. This systematic approach allows for the pinpointing of high-risk areas that demand immediate attention and remediation efforts. Well-established methodologies such as the Common Vulnerability Scoring System (CVSS) serve as a standardized framework for the assessment of vulnerabilities, factoring in elements like base score, temporal score, and environmental score [94]. Furthermore, emerging strategies harness the power of machine learning algorithms and threat intelligence to refine the accuracy of prioritization [95, 96]. Recent scholarly investigations highlight the necessity for dynamic and context-aware vulnerability management tactics [97, 98]. These advancements underscore the evolving landscape of vulnerability prioritization and underscore the paramount importance of integrating state-of-the-art methodologies into an organization’s cybersecurity endeavors.

3 Research challenges and open issues

In this section, we discuss the various challenges and issues that are faced in performing vulnerability detection on embedded device binaries.

3.1 Reverse engineering

Reverse engineering of firmware is a very complex task that involves a series of steps with appropriate tools and expertise. Reverse engineering consists of firmware extraction, firmware unpacking, and firmware disassembly. One of the main problems in the extraction of firmware is the use of appropriate hardware and software. Moreover, these extraction tools are very costly and are often very complex and buggy. Embedded devices are designed without any common standards. Lack of standardization in hardware architectures across the wide range of IoT & embedded devices presents a big challenge in the extraction of their firmware.

3.2 Firmware disassembling

Software programs are cross-compiled and deployed on various architecture platforms which puts a huge challenge on the analyst to disassemble and make sense of the different binary instruction formats of specific architectures which have been compiled from the same source code. It is a very challenging task for the researchers to look at all the binary formats for common vulnerabilities.

3.3 Detection accuracy

Obtaining higher accuracy for the detection of vulnerabilities and reducing the false positives is very critical in vulnerability analysis. Out of all the framework types Automatic analysis by Costin et all [85] provides better results in identifying both known and unknown types of vulnerabilities. Accuracy can be improved by tailoring ML & DL algorithms for such problems.

3.4 Scalability

Vulnerability detection at a large scale is a major challenge. IoT & Embedded devices are growing exponentially due to this the vulnerability detection frameworks have to accommodate these ever-growing devices. Testing the embedded firmware in runtime weather on real devices or through emulation tools is very slow and error-prone. The deployment of different architectures and software programs presents a major challenge in vulnerability analysis.

3.5 Vulnerability verification

Verification of the identified vulnerabilities is another problem that researchers are facing. Verifying requires determining the execution path in a firmware that triggers the vulnerability. Due to the limited information of the vulnerability many times it becomes complicated to verify the vulnerability by reproducing the behavior of the system.

4 Proposed model

We have designed a hybrid vulnerability analysis framework including the testbed. This proposed framework addresses some of the issues present in the already existing frameworks. It addresses the scalability issue by utilizing both the firmware collection methods which include the web crawler approach and extraction using onboard debug ports. Our approach utilizes both dynamic and static analysis techniques for the identification of known and unknown vulnerabilities. QEMU emulator will be used in run-time dynamic analysis of already extracted firmware stored in the firmware repository. If any problem is faced during extraction of firmware using onboard JTAG/UART ports then the firmware will be downloaded using a web crawler as utilized in firmadyne framework. Network Analysis module will be used to check protocol vulnerabilities in IoT devices in runtime using tools that include Wireshark and Metasploit scripts. All the identified vulnerabilities will be stored in the vulnerability repository. The verification process of the vulnerabilities will be carried out by using Machine Learning techniques for the generation of test cases and executing the sequence on emulated firmware or on real devices whichever is feasible. The proposed model will be implemented on a developed testbed.

The proposed testbed as shown in Fig. 16 is a four-layered architecture model. The four layers are the Internet Layer, Control and Monitoring layer, Access Layer, and the Device layer. Internet Layer provides internet connectivity through LAN network using appropriate switches. The monitoring and control layer consists of workstations and a high-performance analysis machine that would be used to perform computationally intensive analysis. It also consists of a control machine that would be used to launch scripts and programs necessary for analysis. All the test results will be stored on this machine itself. The Access layer consists of various hubs and routers that connect wirelessly to the IoT devices. The device layer consists of various IoT and embedded devices that are connected to their appropriate hubs and wifi routers (Fig. 17).

Fig. 16
figure 16

Proposed vulnerability analysis model

Full size image
Fig. 17
figure 17

Proposed vulnerability analysis testbed for IoT and embedded devices

Full size image

5 Conclusion & further work

In this article, we surveyed various types of architectural elements, firmware extraction methods, and various types of vulnerability analysis frameworks for IoT & Embedded devices. We surveyed the major processor architectures of embedded devices. Techniques used to implement static, dynamic, and hybrid analysis was surveyed. A detailed comparison of the vulnerability analysis framework was presented based on various qualitative and quantitative parameters. Finally, we discussed the various challenges in performing vulnerability analysis of IoT devices. A vulnerability analysis model for overcoming some of the challenges is also proposed at the end. As further work, we intend to develop the proposed framework in the lab using various COTS modules. The proposed model will be evaluated based on various parameters on various IoT & Embedded devices available in our institute IoT lab.

In order to solve the lack of standardisation in hardware architectures, future research should focus on the creation of user-friendly and affordable methods for firmware extraction. Innovative methods for deconstructing cross-compiled software programmes and the improvement of machine learning algorithms for vulnerability research are other crucial areas for development. As the IoT ecosystem expands, researchers should concentrate on developing scalable frameworks and procedures for efficient vulnerability verification.