1 Introduction

The industrial environment has undergone massive changes since the First Industrial Revolution, with the latest revolution being called Industry 4.0 or the Industrial Internet of Things (IIoT). Emerging technologies and implementations such as the Internet of Things (IoT), Artificial Intelligence (AI) and 5 G networks have converged with traditional Operational Technology (OT) protocols and devices to improve the performance and efficiency of the enterprise [1]. Some recent proposals from the research community [2] have even started mentioning Industry 5.0, which has the same basic goal of solving the problems related to the integration of IIoT into traditional industry, but while the approach followed in Industry 4.0 has been to implement an independent integration for each company, focusing on the advantages and the functionalities of the technologies related to IIoT, Industry 5.0 adopts a different focus, trying to highlight the importance of innovation and research to support the industry in its service to humanity. Thus, Industry 5.0 pays particular attention the points related to resilience, sustainability and human-centric values, with the support of the technologies considered in Industry 4.0 adding the advances in biotechnology and renewable and energy-efficiency technologies [3].

One of the proposed solutions is Multi-Access Edge Computing (MEC), which has been created as the successor to Edge Computing and is designed to bring the advantages of cloud computing closer to companies, and to give support to emerging technologies. Its main objective is to improve the network performance of cloud applications, as well as to enable the implementation of new applications that are delay-sensitive, such as autonomous driving or virtual reality applications. In addition, MEC facilitates the implementation of IIoT since the concept itself already brings together some of the technologies considered in it [4]. However, the convergence of OT protocols and Information Technology (IT) protocols, together with the inclusion of IoT applications and web applications, can produce cybersecurity risks if this process does not follow the good practices required in implementation. Moreover, the risks are higher in critical infrastructures, which could increase losses for companies and states and have a debilitating effect on sectors such as security, national economic security, national public health or safety. Another aspect to take into account is that many devices found in these infrastructures are not likely to be updated periodically [5]. As a result, these infrastructures are a perfect target for attackers and their strategies, as their techniques change and evolve rapidly. This insecure context can be found in others environments, such as industrial plants, private health care, or industries related to smart city functionalities, each of which has specific requirements that are different from those of critical infrastructures [6, 7]. Thanks to the use of Machine Learning (ML) and AI algorithms, cybersecurity researchers can use the data left by attackers or researchers themselves on their network or application to train multiple tools. As a way to monitor and avoid (or reduce) the impact of these attacks, these tools are becoming the perfect solution for the cyberdefence department of companies. Recently, Federated Learning and Adversarial Networks have been improved in order to use the data collected by the devices in the network, and then train the models with them, which results in these models being more sensitive to changes and new attacks [8]. Unfortunately, evaluating the feasibility and performance of these threat detectors when deployed in low-resource environments such as IIoT is not straightforward, since an architecture is needed for generating reproducible experimentation scenarios while taking into account the specific characteristics of IIoT environments.

There are some related works that have tried to address this problem. The first one is openLEON [9], which is used as the basis of our proposal. However, openLEON does not implement any service related to IIoT. Another interesting proposal is the IIoT Testbed [10], which is an emulator focused on the deployment of an IIoT application to monitor different metrics. The limitation of this tool is the low level of flexibility in its scenario, as it is focused only on the IIoT application deployed, thus making it impossible for use in the study and modelling of IIoT networks and the related experimentation, and it requires a tool to design and deploy networks that follow the IIoT paradigm.

With these issues in mind, this paper presents MECInOT, an emulator built on top of OpenLEON [9] for facilitating the modelling and deployment of an IIoT topology. MECInOT allows the deployment of OT and IT applications up to the edge data centre topology. MECInOT has also been designed with special attention to cybersecurity research, providing different tools for the proposed scenarios in which different attacks on the network and application layers can be carried out. In this regard, we use MECInOT to collect network data while attacks are taking place in the emulation of the scenarios, and then to implement an Intrusion Detection System (IDS) based on different ML algorithms. The use of ML in threat detection offers an advantage over the use of rules found in most commercial IDS. These advantages are due primarily to the inability of rule-based systems to detect unknown or new attacks, in addition to the difficulties that such systems have in operating in highly dynamic environments such as IIoT scenarios [11]. Therefore, the trend is towards the machine-learning-based approach, and in this work we intend to implement the IDS to validate our emulator in the field of cybersecurity research by adopting such an approach.

MECInOT, which is the emulator present in this work is available in a GithubFootnote 1 repository, and the dataset used is also available onlineFootnote 2.

The rest of this paper is organised as follows. In Sect. 2, we review the proposals from the research community regarding emulators for MEC, the IoT and the IIoT. Section 3 describes the technical background of our study. The emulator developed in this experiment is presented in Sect. 4, together with the applications for each OT, IoT and IT protocol and the attacks implemented on it. In Sect. 5, we evaluate the performance of the proposal, using the CPU usage parameter as a reference, and we present the attacks implemented on the emulator and the generation of malicious data. By using these data, in Sect. 6 we describe the methodology followed to make an IDS using ML algorithms and analyse the results obtained by it. Finally, Sects. 7 and 8 correspond to the conclusions that can be drawn from this experiment and the future work planned, respectively.

2 Related work

This section presents a study of the most important pieces of research focusing on the emulation of IIoT and MEC scenarios.

A work which implements an IIoT context in its design is the IIoT Testbed emulator [10]. This emulator allows the emulation of IIoT applications based on Data Distribution Service (DDS) middleware, allowing the users to manage different industrial processes, including the modification of multiple parameters, such as Quality of Service (QoS), of the different services and functionalities emulated. The creation of new IIoT processes and their management is performed by using a web interface, making the use of the emulator easier for the users without the need to understand the whole architecture of the emulator. However, the IIoT Testbed is a very limited tool in terms of its functionality and flexibility when it comes to creating new conditions for the scenario that it proposes. This is mainly because the tool is focused on the management of the process and its related data. In fact, the IIoT Testbed emulator cannot success fully meet the needs of different experiments in creating multiple scenarios and IIoT topologies with different devices and protocols used by the applications. Also, MEC is not considered during the design and implementation of this emulator. This is the case of rest of the proposals described in this section, but in the opposite sense, that is to say that Edge Computing is considered but industrial protocol applications are not.

A work that considers MEC in its proposal is [12], which establishes a development environment for the deployment of applications based on Fog Computing and MEC architectures. This emulator allows the creation, design and definition of networks, the deployment of multiple edge nodes, and the implementation of IoT applications. Moreover, the emulator allows the analysis of the performance of each node deployed and of the network that interconnects them, in order to check whether there are any errors or misconfigurations in the scenario defined or the performance of the application deployed. However, to execute Fogify it is necessary for the user to have an in-depth knowledge of how to use it, as well as some fundamentals on the tool in charge of its deployment, namely Docker Swarm. In addition, this type of tool can lose the support of the community in favour of other popular ones such as Kubernetes. Furthermore, the emulator does not support the inclusion of OT protocols or new IT or IoT protocols, making it impossible to create a heterogeneous IIoT scenario.

Using the emulators Containernet and Maxinet, which are extensions of the Mininet emulator, [13] presents a solution for the creation, design, definition and deployment of virtual networks that users consider. These emulators use the Python programming language, which helps the users with the deployment of the networks through the creation of a single script. Containernet deploys the final devices as containers, providing the flexibility of this technology when creating for example, applications, device or server functionalities. The main problem of Fogbed is that the emulator does not include support for mobile networks, which is one of the principal characteristics of the MEC paradigm. Also, the emulator is not IIoT-focused, making it impossible to implement an MEC-IIoT scenario using just this emulator. In addition, Fogbed uses the default version of Containernet, which does not allow redundacy in the MEC topology, with all that this implies, such as the errors that could appear with disconnections between edge nodes when the complexity of the topology and the number of applications and services increase.

The last emulator found in the literature that can allow the deployment of these kinds of networks is openLEON [9]. This emulator solves the problems found on Fogbed emulator, because openLEON implements a network controller that allows the use of spanning-tree in the topology, thus making it possible to implement redundancy between network devices and edge nodes. In addition, the emulator enables the use of srsLTE, which together with the corresponding hardware allows connectivity with Long-Term Evolution (LTE) networks in MEC topology. Therefore, openLEON is considered one of the most appropriate emulators for use in the deployment of MEC-IIoT scenarios. However, as was mentioned with Fogbed, openLEON is only focused on the deployment of the MEC topology, and IIoT applications and devices are not considered. This means that experimentation, research and security checking in the context of IIoT are impossible for users. In order to address this shortcoming, our MECInOT proposal uses openLEON as a basis for MEC topology deployment, while integrating the deployment of IIoT networks with their corresponding services, applications and devices. This is an added functionality that to the best of our knowledge, has not been included in any proposals in the literature.

As far as we know, there are no other works that address the emulation of MEC-IIoT environments. This fact indicates the important contribution that our work can make to the state of the art. Also, MECInOT introduces new functionalities and characteristics such as MEC integration, flexibility to introduce new IIoT protocols and IIoT services, and the integration of real devices and physical resources in the topologies.

As a summary, Table 1, shows the characteristics of the different pieces of research reviewed in this section.

Table 1 Summary of the proposals from the research community
Full size table

3 Background

In this section some technical concepts related to the emulator and machine learning algorithms are described.

3.1 Multi-access edge computing

MEC, a concept standardised by the European Telecommunications Standards Institute (ETSI), is considered the evolution of Edge Computing and is designed to improve the performance of communications in the Cloud Computing environment and IoT and IIoT technologies. One of the most important characteristics of MEC is the use of virtualisation technology to provide computational resources to applications and the heterogeneous network management that MEC must support [4]. As it brings computational functions closer to end users than Cloud Computing, it makes it possible to reduce the latency in communications and avoid network saturation in the Cloud Computing providers. This means that users perceive a better performance in the applications, and it allows the development of new applications that previously were impossible with such limitations. Also, with the use of the new 5 G mobile networks and the future 6 G ones [14], a reduction in latency between devices can be achieved, making it possible to deploy real-time applications on this kind of architecture.

To make it possible to implement MEC, it is necessary to use multiple virtualisation technologies, namely:

  • Network Function Virtualization (NFV). The implementation of IIoT in traditional factories means that a large number of IoT devices and applications are connected to industrial environments. Many of these devices use different communication protocols, which in some cases will be proprietary, because these devices are designed for specific contexts and scenarios, which in turn means using specific network devices to manage the proprietary protocol traffic. NFV allows network admin to avoid the use of these specific network devices to manage the heterogeneous and proprietary traffic, reducing the cost associated with the deployment of the different network devices and their number, thanks to using generic network devices. Basically, NFV is software that can be installed on any device that can receive and send traffic. ETSI defines and determinates the applications that NFV can support, such as connectivity functionalities, Dynamic Host Configuration Protocol (DHCP) or Network Address Translation (NAT) [15].

  • Software-Defined Networking (SDN). Traditionally, in communication networks the data layer is defined as the management part of the network of the traffic generated by the users, while the control layer manages the routing processes. Both layers work together on the network devices, which produces a slow-down in high-demand networks. The growth in complexity of the networks has shown that this solution is neither scalable nor flexible. To partially solve this issue, network administrators manually reconfigure network devices by using scripts, which leads to misconfigurations, but in any case this becomes an immeasurably difficult task in complex networks [16].

    In view of these misconfigurations from network administrators, SDN tries to solve this problem by disassociating the two layers. Thus, a specific network architecture is defined, and the allocation of the network resources is established thanks to network virtualisation [17].

  • Network slicing. This technique defines different virtual networks depending on a criterion, and the allocation of the network resources is determined according to the needs established for each virtual network. Network Slicing allows the generation of multiple custom networks using the network resources deployed physically. This provides flexibility when allocating the network resources according to the needs of the virtual network, or even if then occurs an unexpected demand for resources at a particular time [4].

  • Service Function Chaining (SFC). The transition from traditional networks to software-defined networks and virtualisation is a huge effort for IT teams [18]. Thus, the principal goal of SFC is to facilitate this transition dynamically. The concept of SFC is quite similar to NFV, the principal difference is that SFC solves the problem that can appear when a service is provided individually, and it belongs to a service chain. Thus, SFC manages the services that are running on the network devices and which can be used at a specific moment [17].

Figure 1 shows an example of implementation of a MEC architecture using the basic NFV and SDN services explained above, illustrating how communication between the SDN controller and the NFV controller is performed in this architecture.

Fig. 1
figure 1

Reference MEC architecture [4]

Full size image

3.2 Machine learning algorithms

Machine learning has grown rapidly in recent years in the context of computing and whole data analysis performed via other technologies such as the IoT. ML uses this new information generated by analysis to develop applications with new functionalities in an intelligent manner. Thus, ML provides the system with the processes and tools to automatically extract knowledge from the data that it receives without the need to be programmed [19].

As mentioned above, we use a machine learning approach because several papers consider it to be best in environments where there are unknown threats, especially if these environments are highly changeable and heterogeneous, due to the difficulty of manually establishing rules for each type of attack. There are comparisons that demonstrate the superiority of machine-learning-based approaches in these contexts [20, 21]. In addition to the better performance of a machine-learning-based IDS, it also offers certain advantages, such as:

  • Accuracy: Machine learning algorithms can analyse large amounts of data and identify patterns and anomalies that may be difficult for humans to spot or define explicitly.

  • Flexibility: Machine learning-based IDS can continuously learn and adapt to new types of attacks and changing network environments.

  • Anomaly detection: By modelling the normal behaviour of the system, it is possible to detect anomalies that are completely impossible to detect for rule-based systems.

Supervised ML techniques are considered to be those algorithms which provide intelligent models with the construction of general hypotheses and patterns [22], using external data instances to address the prediction of future instances from a similar data input. The principal function of these techniques is to develop a prediction model for solving classification and categorisation problems, establishing the possible category of the incoming data [23]. This can be solved with binary classification in the case of there being only two categories to predict, or multiclass classification, which normally refers to the prediction of more than two categories. One must also consider the special case in which the incoming data can be categorised into multiple categories at the same time [24].

Some algorithms that give good results in multiclass classification problems with tabular data are the following:

  • Decision Tree (DT). DT is a well-known supervised ML algorithm [25], and it is also used for regression problems. This algorithm works by using tree structures, starting from a root node and sorting down to certain leaf nodes, whose number depends on the data and the categories. DT performs the classification with the instances, which are classified by checking the attribute established and defined at each node of the tree until reaching a leaf node which shows the category of the entry data. The splitting is carried out by using two different criteria, namely gini and entropy, whose mathematical equations are given by Eq. 1 and Eq. 2 [26].

    $$\begin{aligned}{} & {} Entropy: H(x) = - \sum _{i=1}^{n}p(x_{i})log_{2}p(x_{i}) \end{aligned}$$
    (1)
    $$\begin{aligned}{} & {} Gini(E) = 1 - \sum _{i=1}^{c}p_{i}^{2} \end{aligned}$$
    (2)

  • Random Forest (RF). The RF classifier is an ensemble classification model used in various areas of application [27]. This algorithm makes a parallel ensemble fit many DTs in parallel using different datasets or subsamples of the same dataset to train them. The output is the majority solution of the trees or the average result [26].

  • Naive Bayes (NB). The NB algorithm is based on Bayes theorem, which makes the assumption of the independence between each pair of features that are allocated to the entry data of the model, and its definition is given by Eq. 3 [28]. The principal advantage compared with other approaches is that NB only needs a small amount of training data to quickly estimate the parameters needed. However, the main problem of this algorithm is that its performance can be affected by the strong assumptions of feature independence [26].

    $$\begin{aligned} P(A\Vert B) = \frac{P(B\Vert A) \cdot P(A) }{P(B)} \end{aligned}$$
    (3)

  • Stochastic Gradient Descent (SGD) [29]. SGD is an iterative algorithm for the optimisation of the results with an objective function with appropriate properties. This algorithm allows a reduction in the computational cost of training, especially in high-dimensional optimisation problems, allowing faster iterations at the cost of a lower convergence rate. SGD is usually applied to problems of text classification and natural language processing. However, the algorithm can produce worse results for feature scaling and needs the hyperparameter tuning of some parameters, such as the number of iterations and the regularisation parameter.

  • Support Vector Machine (SVM) [30]. SVM is a supervised ML algorithm used in various fields and kinds of problems, especially classification problems. SVM uses statistical learning approaches and classifies the input data by determining a set of support vectors. The main goal of SVM is find the optimal hyperplane for the classification of new data. In our work, the SVM model developed is optimised with SGD.

There are other possible state-of-the-art algorithms, such as neural networks and their different architectures [31]. However, in classification problems with tabular data they present certain disadvantages, especially in IoT and IIoT environments, where there are resource constraints. These limitations are as follows:

  • Complexity: Neural networks can be complex models to train and require considerable computational resources. This can make them difficult to implement and run, especially for large datasets.

  • Overfitting: Neural networks are prone to overfitting, especially when working with small datasets. This means that they may perform well on the training data but may not generalise well to new data.

  • Lack of interpretability: Neural networks can be difficult to interpret and it can be hard to understand how they make predictions. This can make it difficult to understand the factors that are driving the predictions and to identify any potential biases in the model.

  • Time-consuming: Training a neural network can be time-consuming, especially for large datasets. This can make it difficult to use neural networks in real-time applications, where quick predictions are necessary.

  • Require more data: Neural networks generally require more data to achieve good performance than other machine learning algorithms. This can be a disadvantage if the dataset is small or if there are certain types of data that are difficult to collect.

Overall, while neural networks can be powerful tools for tabular data classification, they can also be complex and time-consuming to work with and may not be the best choice in all cases. Other machine learning algorithms, such as decision trees or random forests, may be more suitable in certain situations [32].

For these reasons it has been decided not to validate the emulator with neural networks, although future experimentation in which the architecture is validated with deep learning architectures of low computational cost would certainly be of interest [33].

4 MECInOT proposal

This section provides the details of the design and the methodology that have been followed in developing MECInOT.

4.1 Design

When designing MECInOT, we considered that the emulator should include the following features:

  • Offer a realistic emulation of physical scenarios. It should allow to users to carry out experiments and proofs of concepts, and to obtain data in the same way as when using a real topology, without the need to use physical devices.

  • Provide the possibility and flexibility to develop different scenarios depending on the research needs. Thanks to the use of the virtualisation of devices using containers, it is possible to easily modify the number of devices, their functionality, or even the definition and deployment of multiple virtual subnetworks, with each one having a specific function or implementation.

  • Facilitate the insertion of real devices in emulated scenarios. The possibility may arise in an experiment of having to use real devices in an emulated scenario in order to validate the performance results. In this proposal it is possible to introduce new metrics such as collisions, packet delays or device performance. Therefore, thanks to the virtualisation technologies used by MECInOT, real devices can be integrated into the scenarios deployed by the emulator, new network devices such as IoT gateways can be implemented. Also the integration of physical radio for specific protocols such as Zigbee, Z-Wave, or 6lowpan is available.

  • Enable cybersecurity research. The main goal during the design of MECInOT was to develop an emulator that make it possible to carry out cybersecurity research in the MEC-IIoT context. Consequently, the emulator provides the users with different scripts and a malicious container that allow them to easily perform different types of attacks in their scenarios, generate malicious data, and check the impact that these attacks have on the topologies. Also, through the design of the emulator, these attacks can have a consequences on the real devices connected to the emulated scenarios, providing more options for validating and obtaining results.

Figure 2 shows the MECInOT architecture, in which we can see the different functionalities considered during the design of the emulator.

Fig. 2
figure 2

MECInOT architecture

Full size image

4.2 MECInOT deployment methodology

In a real Industry 4.0 scenario, it is mandatory to deploy each device of the industrial network or enterprise network, and provide them with an individual network configuration that covers the specific needs of the scenario. In MECInOT, the first step is to virtualise the machine which creates, manages and deploys the industrial private network, as well as the machine which deploys the MEC architecture. Next, the communication between the industrial network and the MEC architecture is defined. To make this possible, both the machines must have the IP address of the same private subnetwork, configuring the network adapter of the virtual machines in bridge mode.

It is necessary to define the subnetwork addresses that are going to be used by the virtualised IIoT devices and the MEC virtual subnetwork. In MECInOT, the emulated IIoT devices are deployed using the container technology tool Docker-Compose, which facilitates and speeds up the deployment and the implementation of changes in the virtual scenarios. For this reason, the configuration of the virtual IIoT networks is given by this tool, which allows the users define multiple subnetworks for each scenario. For MEC architecture deployment, the network configuration used is the default one established by the openLEON emulator with multiple Edge servers that will run the multiple IIoT services of the scenarios. In addition, it is necessary to establish a correct routing configuration on the IIoT network machine in order to enable correct communication between IIoT subnetworks and the MEC subnetwork. Once this process has been carried out, the communication between IIoT and MEC hosts should be tested to check whether it is correct.

The last step consists in deploying the MEC-IIoT architecture in which the experiments are going to be performed. This architecture is comprised of the IIoT devices, which are already implemented in the emulator scenarios provided, and the network topologies described in the following sections.

Figure 3 shows the communication the between different steps mentioned above.

Fig. 3
figure 3

MECInOT deployment methodology

Full size image

4.3 IIoT topology

With the purpose of emulating the network of an Industry 4.0 factory, MECInOT implements a business network using the interface created by Docker-Compose to deploy the different Docker containers, each of which corresponds to an IIoT device in the virtual factory. In addition, the emulator allows the users to define and implement different subnets with the most common protocols that can be found in an IIoT scenario, with these being:

  • IoT protocols. MECInOT includes IoT applications that rely on the most commonly used protocols in this context: Message Queuing Telemetry Transport (MQTT), Constrained Application Protocol (CoAP) and Advanced Message Queuing Protocol (AMQP) [34, 35].

  • OT protocols. The protocols added to MECInOT are versions of the industrial protocols which use the TCP transport protocol: Modbus/TCP, which is the most widely used industrial standard and protocol [36]; S7COMM, which is the proprietary communication protocol used by the Programmable Logic Controller (PLC) of Siemens [37]; and Open Platform Communications United Architecture (OPC UA), which is considered as one of the protocols that allows the convergence of OT and IT protocols [35].

  • IT protocols. This group of protocols includes those that are widely used by Internet users, such as the Hypertext Transfer Protocol (HTTP) [38].

As mentioned above, the design of MECInOT is focused on enabling cybersecurity research in MEC-IIoT scenarios. Consequently, the emulator provides a set of tools that allow users to carry out attacks in these scenarios, thus making it possible to evaluate the impact of such attacks, as well as the risks and the costs that these could entail in the particular context studied. This toolset is currently built into an attacking node using the Kali Linux distribution image which ca be run in a Docker Container on the IIoT network.

To evaluate the impact of a possible attack on the infrastructure deployed with the emulator, different types of attacks are employed. These attacks belong to the following categories:

  • Packet manipulation. To carry out this type of attack, the attacker node must use the Man in the Middle attack to capture the packets and modify them. The changes made to the packets are focused on the data field of each protocol in the emulated scenario. The rest of the fields do not change.

  • Brute force. The attack uses a dictionary of users and passwords to make multiple login attempts to try and access devices and services. Thus, MECInOT has a script that performs the dictionary brute force attack on a login form of a web server that can be running on a PLC or edge node.

  • Attacks with payloads in HTTP frames. These attacks aim to exploit the Shellshock vulnerability [39], which is a software bug found on some web servers that can be used to gain access to the machine on which the service is running. A script is made to automatise the process of gaining access to the machine shell. This vulnerability is exploited by sending an HTTP packet with a modified User-Agent field that contains the specific payload.

  • Network scanning. The emulator includes a tool that allows different scanning methods to be carried out automatically. The scanning methods implemented are TCP SYN, TCP connect, UDP, TCP NULL, TCP FIN, TCP XMAS, and TCP ACK.

  • Denial of Service (DoS). For this type of attacks, traditional denial of service methods have been implemented, such as Ping of the Dead [40], and methods based on the flood or saturation of the port with a massive sending of TCP packets. In addition, these methods have been adapted depending on which protocol is targeted. In particular, for the AMQP protocol, fake devices are included in order to overflow the message queue, resulting in legitimate user not being able to establish communication with the queue. With regard to CoAP protocol, this uses UDP as a transport protocol, which allows the implementation of the UDP amplification attack [41]. This attack consists in using a simple modified request to a method on a CoAP server to be received by the target machine. As the attacker sends lightweight packets, this makes it possible to cause a denial of service for some of the devices in the network using the CoAP server, which sends larger packets than the attacker.

  • Malicious device injection. In this attack, the attacker tries to find a default configuration or a bad implementation of an MQTT broker in order to obtain more information from it. This process is automatised by using a script which emulates a new device that tries the connection to the specific topic on the MQTT broker as a way to read whole messages that pass through it.

4.4 MEC topology

As was mentioned above, the MEC topology is deployed using the openLEON emulator. In this emulator there two key components:

  • Data centre. openLEON implements this part of the topology by using the Containernet emulator [42], which is an extension of the Mininet emulator [43], thus allowing the creation of topologies whose hosts are implemented with containers. Secondly, the architecture of the topology has a 3-level hierarchical network structure, which is typical of a 3-tier data centre. The topology has two core switches, and two aggregation switches for each one. Also, the architecture has 64 hosts connected between the switches found on the Top of Rack level. This topology structure provides redundancy between network devices and hosts, thus trying to avoid communications errors in the network. In order to do this, it is useful to implement a spanning-tree protocol [44] on the network devices. Since the default SDN controller implemented in Mininet does not support this functionality, openLEON developers decided to implement and use RYU [45], which is a module implemented with Python that provides support for OpenFlow to be integrated in the topology.

  • Mobile communication. As was mentioned in Sect. 2, this is a crucial MEC component. The protocol used to provide mobile communication is LTE, and this is achieved by using the srsLTE emulator [46]. In order for this emulator to run correctly it needs a set of hardware, such as antennas and mobile LTE stations. With this module it is possible to establish communication between the mobile devices connected via LTE with the MEC-IIoT topology.

4.5 MECInOT distributed deployment

With the aim of taking advantage of the container and Mininet virtualisation technologies, a distributed deployment is also defined in order to be able to use MECInOT on a High Performance Computer (HPC) or cluster, and thus obtain a more realistic and better performance than when using a single computer. This kind of deployment allows the use of multiple MEC and IIoT topologies on the distributed nodes of a cluster. For this implementation, it is recommended to use a container orchestrator for the management of the containerised IIoT topologies. In addition, it is possible to deploy multiple MEC topologies without the need to use the virtual machine by only employing the Containernet implementation of openLEON.

Fig. 4
figure 4

MECInOT logical distributed architecture

Full size image

Figure 4 presents a logical implementation of this type of deployment, also showing the intercommunication between the nodes and topologies providing a correct functionality and connection between the different parts of the emulator.

5 MECInOT evaluation

In this section some examples of scenarios that can be designed and deployed with MECInOT are described. In addition, we present an evaluation based on the cost of CPU usage metric for the physical machine on which the emulator is run. Finally, a cybersecurity analysis and proof of concept are carried out to check whether the emulator can contribute to this research field with the MEC-IIot scenario deployed.

5.1 Hardware setup

For the evaluation of the proposal described in Sect. 4 we used a laptop with an Intel i7-10875 H 2.30GHz CPU with 32 GB RAM memory, running Windows 10 20H2. In order to execute the virtual machine, the hypervisor VirtualBox 6.1.16r was used. Finally, a Raspberry Pi Zero 2 was included as additional hardware in order to add the possibility of deploying an IoT gateway in our scenarios.

5.2 Industrial OT scenario

In order to evaluate this scenario we only deployed devices that specifically use industrial or OT protocols, as mentioned in Sect. 4.3.

The scenario studied for the OPC UA protocol consisted of a client node which was in the industrial topology, and a server node that was allocated in the MEC topology. The basic functionality between the two nodes was a communication based on the reading of a random string that was generated by the server node at a random time between one and nine seconds.

For the case of the S7 protocol, two client nodes were deployed, one for reading and the other for writing, and these nodes were allocated in the industrial network. A server node was also deployed in the scenario, but it belonged to the MEC topology. The functionality provided by these nodes was very similar to that mentioned for the OPC UA protocol. The writing client wrote a random string on the server, and then this was read by the reading node at a random time.

For the ModBus/TCP protocol, the deployment involved two clients allocated in the industrial topology, one for writing and the other for reading, and a server node that was deployed in the MEC network. In this scenario, the writing node wrote a sequence of values, which could be alternately True or False, to the server, with the reading node periodically reading the values stored on the server.

In order to provide the emulator with cybersecurity testing capabilities, an attacker node is deployed in the industrial topology, and it can communicate with the rest of the industrial devices and MEC servers. This node, as well as the tools, scripts and attacks implemented, is described in Sect. 4.3.

5.3 Industry 4.0/IIoT scenario

This scenario is an extension of the scenario detailed in Sect. 5.2, and introduces communication between devices using IoT protocols and the emulation of users that operate with IT protocols. This allows researchers and emulator users to implement and deploy an example of an Industry 4.0 factory, which is connected to different network services. In this scenario, the IoT traffic passes through an IoT gateway which is deployed in the business network. In addition, an attacker node is introduced in order to provide the user with the option of performing cyberattacks with the new devices included.

The MQTT protocol is implemented with the aim of emulating a factory that has multiple sensors sending the temperature in degrees Celsius every three seconds. The IoT gateway is given the role of MQTT broker, acting as an intermediary between the communication with the MQTT subscriber, which belongs to one of the edge nodes in the MEC network, and the rest of the devices.

In the case of the CoAP protocol, a client is deployed in the industrial network and it sends multiple messages to the IoT gateway, which performs the function of master/server node, with these messages being read by another CoAP client allocated in MEC topology.

For the AMQP protocol, a client node is deployed in the industrial network and it publishes a random number between 33 and 126 in the RabbiMQP queue that is running on the IoT gateway. These messages are received by an AMQP client allocated in the MEC topology.

The IT protocols that are considered and included in the scenario are the HTTP and Hypertext Transfer Protocol Secure (HTTPS) protocols. The former implements a login form for the factory users, and the latter does so with a video streaming server. On the user side, they are specifically in the industrial network and have been implemented using Python scripts that emulate their normal behaviour using the IT applications mentioned above.

In Fig. 5, a logical schema of communication between the different parts in this scenario is shown. In addition, it describes how communication is performed depending on the type of traffic.

Fig. 5
figure 5

Logical network schema of Industry 4.0/IIoT scenario

Full size image

5.4 Performance analysis

Now we have described some of the scenarios that can be created and deployed with MECInOT, we analyse its performance in these scenarios. This analysis considers the cost in terms of computational resources associated with the deployment of the Industry 4.0 scenario on the machine, since it is the scenario with the highest number of devices and services running simultaneously. The metric used is the CPU usage of the host machine running Windows 10, and the tool used to obtain this metric is System Monitor, which is a piece of native software for obtaining information regarding hardware conditions and process metrics.

Fig. 6
figure 6

Evolution of CPU usage with the deployment of the Industry 4.0 scenario. The X-axis represents the running time of the experiment in minutes:seconds format, and the Y-axis represents the CPU usage percentage

Full size image

In Fig. 6, the evolution of CPU usage over nine minutes (the execution time of the scenario) is shown. The metric is measured from the launch of the virtual machines to the stopping of the emulator. We can observe the different load rate changes during the deployment of the different parts of the emulator, and the fact that the maximum CPU usage occurs in the building and deployment of the Docker-Compose containers.

Fig. 7
figure 7

Evolution of CPU usage with the deployment of the OT scenario. The X-axis represents the running time of the experiment in minutes:seconds format, and the Y-axis represents the CPU usage percentage

Full size image

In order to check the flexibility and how load usage varies depending on the complexity of the scenario deployed, a comparison is made with the scenario that only deploys the applications that uses OT protocols and the above mentioned one. In this case, the OT scenario is simpler than Industry 4.0 and allows us to make the comparison desired. Figure 7 shows the evolution of the CPU load usage during five minutes of the execution of the OT scenario.

By comparing Figs. 6 and 7, we can see that the resources needed to run a given scenario vary in accordance with its complexity. However, by measuring the median CPU usage in the two scenarios, it can be concluded that the difference between the two cases is only 2%. This means that it is possible to increase the number of services and devices in the scenarios without generating much higher rates of CPU usage on the host system. In addition, the maximum load usage in the OT scenario occurs during the launch of the virtual machines of MECInOT, while in the Industry 4.0 scenario it occurs during the event associated with the building and deployment of the containers allocated in the industrial network. This experiment demonstrates the viability of deploying multiple scenarios on a system with limited computational resources.

5.5 Cybersecurity analysis

After evaluating the performance of the emulator, a proof of concept was conducted by carrying out cyberattacks in the Industry 4.0 scenario deployed. In this case, for the proof of concept we considered all the attacks implemented and described in Sect. 4.3. Firstly, the manipulation attack was performed, followed by the attacks related to http frames. Then the scanning network tools were run in the scenario, and the DoS attacks were implemented. Finally, the malicious device injection was run to prove its functionality. Figure 8 shows the IIoT baseline scenario designed to explain the attacks implemented in MECInOT. In this scenario, the legitimate nodes are shown with a green background, the legitimate IIoT servers are shown with a blue background, and finally the malicious node, which will introduce the network attacks in the scenario, is shown with a red background. The behaviour of the malicious node in the scenario varies in accordance with the attack that is running in the scenario.

Fig. 8
figure 8

IIoT baseline schema for its cybersecurity analysis

Full size image

5.5.1 Manipulation attack

In order to perform a successful manipulation attack, first the attacker node must execute a Man in the Middle attack using the arpspoof tool with the aim of being able to capture the traffic between the victim node and the destination. The manipulation attack scheme is shown in Fig. 9, which illustrates how the malicious node is allocated in the middle of the communication between the legitimate nodes and legitimate server, and how it returns the manipulated messages to the destination. In MECInOT the manipulation process is carried out using a Python script on the malicious node, which changes the information allocated in the data field in the corresponding IIoT protocol used in the communication. Figures 10 and 11 show how the writer client sends strings of random characters without numbers, while the reader client receives a string of numbers from the server. This means that the attack has been successful and the results obtained are as expected.

Fig. 9
figure 9

Manipulation attack schema

Full size image
Fig. 10
figure 10

Messages sent by the writer client

Full size image
Fig. 11
figure 11

Modified messages received by the reader client

Full size image

5.5.2 HTTP application attacks

Figure 12 shows the schema followed to implement and run the brute force and payload attack against HTTP services. This schema shows how the legitimate users transmit legitimate HTTP traffic while the malicious node introduces a huge number of HTTP frames in the brute force attack and HTTP frames that carry a payload trying to exploit a possible Shellshock vulnerability allocated in the HTTP service.

Fig. 12
figure 12

HTTP applications attack schema

Full size image

The brute force attacks in MECInOT were implemented by using the Hydra tools, which is already installed on the attacker node of MECInOT. To launch this attack, there is a script which runs the tool with the corresponding parameters to indicated in the script and the dictionary used by the nmap tool, which is in charge of the enumeration of the services and passwords. Figure 13 shows a screen capture of the traffic sniffed to check whether the attack has been carried out successfully against a web server with the IP address 10.0.0.1, showing how multiple and continuous HTTP login petitions are sent to the login form.

Fig. 13
figure 13

Traffic generated by the brute force attack

Full size image

In order to launch the Shellshock payload, a python script is used. This script facilitates the sending of the HTTP frames with the payload to gain access to the device with this vulnerability, whose IP address is 10.0.0.1 in the MEC topology. Figure 14 shows an example of an HTTP packet sniffed that contains the payload in the User-Agent field.

Fig. 14
figure 14

HTTP frame with malicious payload in the User-Agent field

Full size image

5.5.3 Network scanning

Network scanning allows attackers to discover active nodes and servers that are running at that moment. Also, the attacker receives additional information from these devices, such as the IP address, open ports or services running. Figure 15 shows the schema for running this attack in the scenarios. The malicious node sends the scanner packets to every possible device running in the MEC-IIoT network and receives the response of each legitimate node in the network.

Fig. 15
figure 15

Network scanning schema

Full size image

For the scanning functionality and implementation for the attacker node in MECInOT, a script was developed that automatises this task by using Python and Python-nmap modules. Figure 16 shows an example of the scanning traffic generated by the script that tries to scan the services running on the MEC servers in the scenario.

Fig. 16
figure 16

Traffic generated by network scanning

Full size image

5.5.4 DoS attacks

The DoS attacks tested in this analysis are the generic ones mentioned in Sect. 4.3, and the schema DoS attack against a legitimate IIoT server in the scenario by a malicious node is shown in Fig. 17.

Fig. 17
figure 17

DoS attack schema

Full size image

To run them, a Python script that allows launching the Ping of Death attack and the Teardrop attack with DNS packets is included. Figures 18 and 19 show the attacks that are launched against the edge server in the MEC topology whose IP address is 10.0.0.1. In these figures it is possible to identify the packets that are generated by the scripts, and which allow the generation malicious DoS traffic in the scenarios.

Fig. 18
figure 18

Ping of Death malicious traffic

Full size image
Fig. 19
figure 19

Teardrop attack traffic generated

Full size image

5.5.5 Malicious device injection

To evaluate the MQTT malicious device injection attack, the following devices are used: an IoT gateway with a Mosquitto broker running, two IoT devices in an IIoT topology which send temperature data to two different topics whose definitions are ’topic1’ and ’topic2’, and the malicious IoT device run by the attacker. Figure. 20 shows the attack schema implemented in the scenario to run this attack with the devices described above.

Fig. 20
figure 20

Malicious device injection schema

Full size image

Listing 1 shows how the malicious IoT device is implemented and how the device tries to subscribe to the # topic. If the broker only has the default configuration, the malicious IoT device will receive all the messages on any topic handled by the broker. Figure 21 shows how the malicious device receives the message from ’topic1’ from the first IoT device and ’topic2’ from the second one.

Listing 1
figure a

Malicious MQTT device implementation

Full size image
Fig. 21
figure 21

Messages received on other topics by malicious device

Full size image

Under these circumstances, it can be concluded that this analysis shows that the malicious data generated in the scenario is suitable for use in the creation of new applications using big data and ML techniques, or simply for its analysis (Fig. 21).

6 IDS based on ML algorithms

In this section, we describe the methodology and workflow followed to develop the IDS based on ML algorithms. In addition, we present an analysis of the results obtained from the algorithms selected.

6.1 Methodology

Fig. 22
figure 22

Workflow of the experimentation

Full size image
Fig. 23
figure 23

IIoT experimentation scenario

Full size image

The workflow for the development and deployment of the smart IDS is shown in Fig. 22.

Network Deployment. The OT scenario described in Sect. 5.2 is deployed for the IDS development. The way in which the OT services are distributed in the IIoT topology and on the MEC edge servers is shown in Fig. 23.

Data Extraction. The raw traffic data are extracted once the scenario and attacks are running. For the collection of network data, Wireshark is used on the SCADA node. Finally, we analyse these network data to extract features considered for the input of the ML models in the following steps. The features extracted from the network packets are shown in Table 2.

Data Preprocessing. In this stage, the data extracted are adapted in order to be used with the algorithms. Firstly, the raw data have to be tagged into the corresponding category, with the aim of enabling the models to classify them correctly. The categories tagged and the number of each one are detailed in Table 3. Moreover, it is necessary to study them to determine which techniques to apply to the data and which features are most relevant. For this purpose, the feature selection is performed with Extremely Random Forest [47]. In addition, some features are combined and retaught to receive greater importance during the training of the models.

Model Training. Once the data are ready to be used with the algorithms, namely DT, RF, NB and SVM Linear+SGD, it is necessary to distribute the data in different dataframes. These dataframes hold benign traffic and one type of attack or multiple attacks. In addition, the dataframes are divided up into 70%, which uses the K-Fold Cross-Validation technique, dividing the data into five subsets to obtain the best model, performing an experiment per subset, while the 30% of the dataset remaining is dedicated to the validation stage, which allows us to identify the performance of the model when new data are introduced after the training.

Model Validation. When the training stage is finished, it is time to validate the predictions of the final model. With the aim of evaluating whether the model correctly predicts the testing dataset, the following metrics are used: Accuracy, Precision, Recall, F1 Score and Training Time. All of these except the last one use the information of the confusion matrix, which is constructed with the number of True Positives (TP), True Negatives (TN), False Negatives (FN) and False Positives (FP) [48].

Thus, Accuracy metric score shows the percentage of predictions made correctly, although this metric does not explain the performance of a model when a class-imbalanced dataset is used as in our experiment. For this reason, in our analysis of the experiment Recall, Precision and F1 Score are also included. The Recall metric score shows the percentage that the model detects for which the packet belongs to attacks but the classification is not correct. A high Precision metric score means that the model does not detect many FP. The F1 Score shows the balance between these 2 scores. The Training Time metric is also included and it shows how much time it costs to train a model with the technique used to develop the model.

Table 2 Features extracted from the network data extraction
Full size table
Table 3 Categories, tags associated and the number of packets for each one
Full size table

6.2 Results

Table 4 Performance results for the models
Full size table

Table 4 shows the results for each metric and algorithm used and for each attack considered for detection by the classification model. The analysis is divided into the different attacks implemented and introduced into the dataset, so that the performance of each scenario can be evaluated.

Packet manipulation attack. This type of attack is normally difficult for classifiers to detect correctly without having a good definition of the features. The results returned by the algorithms show that the best one for detecting this attack is RF, since it achieves a score of 96% in the metrics of Precision, Recall and F1 Score, and 98.33% in Accuracy. However, RF has the worst result in terms of Training Time, and, looking at the results returned by the DT algorithm, it is possible to reduce training time without significantly sacrificing performance, as the differences in the key metrics are around 1–2 %. In addition, when it comes to obtaining better Precision than Recall results, NB is an attractive option, but it has a worse F1 score than DT. Finally, the results returned by SVM Linear+SGD are significantly worse than those of the rest of the algorithms in the study.

Scanner attacks. For the scanners that can be run in the MEC-IIoT scenarios, DT and RF return the same results for the metrics selected. Specifically, the results are 92%, 99%, 94% and 99% for the Precision, Recall, F1 Score and Accuracy metrics, respectively. The training time metric shows that DT is faster than RF, and therefore it is the best algorithm to use in this case. With respect to the NB and SVM Linear+SGD algorithms, they obtain worse results than the algorithms mentioned above. Both NB and SVM Linear+SGD obatin the same results, with 91% in Precision, 96% in Recall and 92% in F1 Score, with the only difference being 0.01% in the Accuracy metric. As can be seen, the difference in the results between DT/RF and NB/SVM Linear+SGD are not very significant.

DoS attacks. In this scenario it is possible to identify two groups of results: the first one, with 100% in Precision, 97% in Recall, 98% in F1 Score and 99.94% in Accuracy, is the one obtained by the algorithms DT and RF. Therefore, the best-performing one in this group, and in general, is DT as it is faster to train than RF. Specifically, DT takes 0.0079 s to be trained, instead of the 0.2750 s that RF needs to finish the training model. The second group returns worse results in general than the first one, with 100%, 79%, 84% and 99.62% in the Precision, Recall, F1 Score and Accuracy metrics, respectively.

HTTP application attacks. The results for the HTTP attacks implemented in the scenario show that all the algorithms achieve 100% in for all the metrics. The reason for this is that the parameters considered for the models are able to detect the attacks easily thanks to the information in the HTTP frames for payload attacks, and the time delay between packets for brute force attacks. Therefore the selection of the best algorithm should be based on another metric, such as the resources needed to run the model or the training time of the models, depending on the needs of each scenario in which the IDS will be implemented. When using the training time it is possible to determine that the best algorithm for this type of attacks is DT, with 0.0079 s.

Mixed attacks. This case is the most realistic scenario that it is possible to find in real Industry 4.0 factories when they are under attack from multiple attackers at the same time. Under these circumstances, the results obtained in this experiment are crucial in order to select the algorithm to be used for the IDS implementation. The best results are achieved by the DT and RF algorithms, which obtain 99% for every metric. The main difference between them is the training time, with a better result being achieved by the DT algorithm. In addition, it is important to highlight the poor results returned by the SVM Linear+SGD algorithm for the Precision, Recall and F1 Score metrics, with these being around 50%. This is particularly curious when the Accuracy metric shows 99.63%, which means that SVM Linear+SGD does not correctly classify the packets and is probably grouping most of them in to a single class.

Finally, an analysis of the metrics shows that the best options for all the testing scenarios are DT and RF, since both of them return similar results. As has been mentioned during the analysis of each case, the main difference between the two solutions is in the Training Time metric. This metric gives an idea of how much time it takes to train the model with a specific parameter, and how it could scale when the dataset size increases. Furthermore, NB produces very similar results to DT with a difference in performance of around 1-6 % in the Packet Manipulation, Scanner, HTTP and Mixed attacks. However, the results obtained in the DoS attacks are significantly worse than the DT and RF algorithms. Lastly, the worst ML algorithm included in the analysis is SVM Linear+SGD, with a performance similar to NB in the Packet Manipulation, Scanning, DoS and HTTP attacks. However, it obtains the worst performance for the Mixed attacks and in all cases returns the longest Training Time.

7 Conclusion

In this work, MCEInOT, a cybersecurity-oriented emulator for the deployment of MEC-IIoT topologies for experimentation in this context has been presented. It provides cybersecurity researchers with different tools for carrying out network and application attacks in any scenario deployed with the emulator, supporting IIoT, IoT and IT protocol-based applications. In addition, it has been shown that our proposal can be used to extract data from network attacks made on the network on which the experimentation scenario is deployed, and then use these data to train different ML algorithms to be deployed as an IDS for the MEC-IIoT topology. The experiment has allowed us to evaluate some ML algorithms for classification purposes, namely DT, RF, NB and SVM Linear+SGD. Although they provide quite similar performances in some situations, the DT and RF achieve the best general results, with the training time metric showing that DT obtains better results than RF.

8 Future work

In this section, we describe several future projects that could improve the emulator, as well as some lines for additional research that could derive from this experiment.

  • Optimisation of the container images. It would be highly beneficial to reduce the size of the containers in the scenario in order to improve the speed and resource consumption during the deployment of the topology of the application nodes.

  • Unification of the emulator on a single virtual machine. In order to properly use the emulator on a single computer, it is necessary to run two virtual machines and interconnect them using a private network. This produces an overuse of computational and network resources that could be avoided if the two virtual machines were joined into a single one.

  • Extend experiments with other ML and Deep Learning (DL) techniques. In this work, a study of 4 ML algorithms for the development of an IDS was carried out. However, there is the option of testing additional ML techniques such as kernel-based algorithms or boosting-tree algorithms. Even some lightweight DL approaches could be considered for integration into MEC-IIoT environments.

  • Study of implementation of the IDS in the scenario. Once it has been decided which ML algorithm to select to implement the IDS, it is necessary to deploy it in the scenario and study its performance when the attacks are running. Therefore, it would be useful to analyse the possible implementations of the IDS given the possibilities offered by the inclusion of an MEC topology in the IIoT scenario. In addition, various IDS architecture proposals can be implemented, using MEC as the principal component to control the traffic or deploy the smart IDS on each device in order to detect possible anomalies individually.

  • Implementing new network functionalities. MECInOT is mainly an emulator oriented for use in the cybersecurity field, so it would be interesting to implement other network functionalities such as firewalls, or topologies with a cyberdefence architecture in mind near the implementation of a Demilitariesed Zone (DMZ), in order to improve the quality and the possibilities during experimentation.

  • Introduction of new applications into scenarios. Thanks to the possibilities that openLEON provides when deploying different MEC topologies, it would be interesting to offer the option of implementing new emerging technologies oriented to wards security and privacy. One example would be the smart contracts based on blockchain to preserve the privacy and integrity of the data.

  • Inclusion of specific attacks for the different protocols. The attacks included and described in MECInOT are specifically designed for carrying out generic network cyberattacks in the scenarios that the emulator can deploy. However, it would be useful to add new types of attacks that are focused on exploiting vulnerabilities found on OT and IoT devices or in their protocols. In addition, these attacks should consider not only IIoT vulnerabilities but also those associated with the services and the virtualisation functions in the MEC paradigm [49].