Keywords

1 Introduction

Digital transformation is the integration of digital technologies in a company to improve performance and productivity. Digital transformations can be applied to any area of the business, and impacts the manufacturing shop floor, the business processes and models used, and the overall customer experience [1]. Digital twins and simulations are increasingly used to plan and optimise processes [2], and given the large volumes of data generated by these, there is a push toward using smart data analytics tools to analyze the stream of data and let managers take responsible, rapid decisions to regulate and improve productivity [3]. Machine Learning (ML) algorithms provide solutions for processing large volumes of data [4], for example, processing and making predictions with sensor signals [5].

By applying simulation and digital twins, it is more realistic to implement reconfigurable systems and customized, demand-based production. The application of artificial intelligence to the digital twin-based streams of data allows companies to analyze and react to sudden changes in demand or disruptions [2]. The reconfiguration of production processes in response to external changes is a difficult challenge. The scope of system reconfiguration includes system layout, process parameters, operation time of multiple assets, sequence of the operations, and material handling systems.

This paper proposes a new framework to enable the system to do different kinds of reconfiguration through the application of existing artificial intelligence algorithms, such as the genetic algorithm and particle swarm optimization (PSO), or newly developed algorithms. With this framework, the manufacturing system reconfiguration can be simulated at first in the virtual environment, and then transferred to the actual production plant. We developed a digital twin environment that simulates the manufacturing process with multiple industrial robots performing multiple tasks. A data pipeline with an API that integrates artificial intelligence is built in the digital twin. With the help of artificial intelligence and the digital twin, the performance of the decision-making process and the reconfiguration process is improved. Finally, a multi-agent program to communicate between different agents and share the negotiation status that determines an optimal configuration for the manufacturing system is made. Section 2 presents a brief overview of works related to the same topic. Section 3 introduces a new framework for manufacturing system reconfiguration and a framework of the reconfiguration engine. Section 4 describes an application with multiple robots, which applies the previously mentioned reconfiguration framework. Section 5 presents the conclusion and suggestions for future research.

2 Related Work

Increasing market demand for mass customization is forcing manufacturers to adopt rapid reconfiguration capabilities, allowing for the sudden change of system configuration – whilst maintaining full system effectiveness – in the event of unpredictable customer demands, line failures or need for maintenance [6]. There is a large body of research on automation agent architectures – distributed artificially intelligent programs that collaborate to solve problems – that allow for ‘on the fly’ reconfiguration of manufacturing systems, exploiting asset flexibility and control to change the behaviour of the manufacturing system [7]. Multi-agent systems and the concept of cyber-physical systems (CPS) were key contributors to research in the reconfiguration of manufacturing environments at the beginning of Industry 4.0 [8, 9].

Emerging artificial intelligence techniques are now being applied to the problem of manufacturing reconfiguration [10]. Human-machine collaboration is an increasingly important concept in flexible and reconfigurable manufacturing systems, and artificial intelligence is also useful here, as well as considerations of safety in such environments [11].

Another important topic that emerged from smart manufacturing’s adaptability to demands and the consequent issues of decision-making based on data is the self-repair ability of smart systems; a recent study handled it again using an artificial intelligence approach to select the best strategy, based on product and module swapping, operation rescheduling and reconfiguration, to significantly reduce the capacity loss [12]. The volume of data coming from this knowledge base is a challenge to handle, but also represents an opportunity exploitable for the development of the Digital Twin (DT) concept, a virtual representation of a physical asset where both counterparts are connected to each other and are dynamically evolving through the whole life cycle [13]. DT emerged as a concept for monitoring processes and is evolving towards being an instrument used for reconfiguration. They can be used in parallel with an artificial intelligence approach to reconfiguring human-robot collaborative assembly lines [14, 15]; they model the real world and exchange data back and forth with the various assets of a manufacturing environment to be able to affect the system’s decisions and behaviour [16]; and they can represent the base on which to build smart, fast and responsive manufacturing systems based on robotics assets [17, 18].

An implication of these proposed approaches is that integrating complex adaptive systems with artificial intelligence techniques, combined with flexible multi-functional manufacturing assets such as robotics, will result in a continuously evolving and changing knowledge base. This volume of data in this knowledge base is both a challenge, but also an opportunity for enabling real-time learning of new reconfigurations and behaviours in manufacturing areas such as production, logistics, and assembly. All the different methods of artificial intelligence aim to achieve the best possible performance in terms of scheduling, efficiency, and productivity, helping to apply autonomous technologies and behaviour to robots, manipulators, and eventually the whole system. Given the increasing availability of data, and the ability to be iteratively applied, this approach is also self-improving [19].

3 Framework

3.1 Overview of Manufacturing System Reconfiguration System

This section proposes a framework for the intelligent optimisation of manufacturing system reconfiguration management. It also provides a concept of the reconfiguration engine with multi-agent system integration. This reconfiguration framework covers two levels; the manufacturing execution level, and the controller level. The manufacturing execution level interacts in real-time with the controller level to deal with the customer’s orders and make decisions. The controller level will receive the information from the manufacturing execution level and send the related control command to the physical device. The reconfiguration engine is a digital environment with multi-agent system integration and enables the system to determine the optimum reconfiguration based on the selection criteria it receives from the decision engine and the reconfiguration data bank (Fig. 1). Layout, cost, time, sequence of the operation, and others can be used as the selection criteria. This framework consists of the following steps:

  1. (1)

    Product requirement

    The first step in the framework is to generate the product requirements, as these will specify the operations the system must perform. The bill of resources and the bill of processes will be generated, breaking the requirements into the equipment needed, and the order of processes required to manufacture the product. This step can be done with several different approaches, including the customer specifying exactly the bill of resources and bill of the process themselves, manually creating the bill of resources and process, or utilising a semantic model and matching algorithm to generate these automatically.

  2. (2)

    Experience data bank

    The data bank is a database for storing data from (i) the current system configuration, (ii) prior experience and configurations, and (iii) stored algorithms and simulations required to support reconfiguration decision making. The current system configuration data (i) consists of information about the current system, such as process information, equipment information, layout information, machine runtime condition, and value-chain information. Standard data models are used to represent the current system configuration information. For instance, asset administration shells can be used to represent system configuration information [20]. The experience repository (ii) is a data bank of historical data for helping the system’s decision making. For instance, configuration information from previous system reconfigurations and processes and their performance. The reconfiguration data bank (iii) contains the offline data components of digital twins for the current use case, the simulation part of the digital twin optimisation algorithms, simulation results from the previous system configurations, and libraries of PLC code for updating the control programs. To store the offline data of the digital twin, multiple approaches are used. For example, the simulation model, the interfaces to access data in the live physical system, and other information such as device information, operation information, and signal information are all stored in the database for future use.

  3. (3)

    Decision-making engine

    The information from the bill of resource/process, system configuration, and the experience repository are analyzed by the decision-making engine. The decision-making engine determines if a reconfiguration is needed based on different aspects, such as cost, time, utilization of the equipment, and productivity. If a reconfiguration is needed, the decision-making engine will give recommendations via an agent to the reconfiguration engine about the reconfiguration approach to be used based on previous successful experiences. The reconfiguration approach can be an algorithm or a combination of algorithms that can be best utilized.

  4. (4)

    Reconfiguration engine

    The reconfiguration engine is used by the decision-making engine to begin the reconfiguration process. Depending on the type of system being reconfigured, different approaches are selected for use, such as inbound reconfigurable transportation systems [21], system layout problems [22], reconfigurable manufacturing system configuration selection [23], or planning & scheduling in reconfigurable manufacturing systems [24, 25]. The optimized approach will be sent to the orchestrator after the reconfiguration has been made. Additionally, the optimized approach will also be stored in the data bank for future reference.

  5. (5)

    Orchestrator

    The orchestrator has a similar function to an ERP (Enterprise resource planning system) and can work with existing ERP systems. The orchestrator is responsible for arranging, controlling, and optimizing workloads in production. The orchestrator will allocate manufacturing resources, plan human resources, and plan and schedule production processes, based on the information received from the decision-making and reconfiguration engines.

  6. (6)

    Multi-agent integration To make the system resilient and robust, the multi-agent approach is used. There are five types of agents in this framework: customer agent, database agent, decision agent, reconfiguration agent and scheduling agent (see Table 1).

Fig. 1.
figure 1

Overview for system reconfiguration

Full size image
Table 1. Agent description
Full size table

3.2 Reconfiguration Engine Algorithm Based on Digital Twin

The reconfiguration engine can select multiple algorithms (either provided by the user or previously stored in the data bank) to determine the optimum system configuration for a given product. In complex robotic manufacturing use cases, many types of optimisation decisions are needed to specify the reconfiguration. These include robot locations, robot paths, and the operation sequence. A digital twin is needed to simulate the process to test multiple scenarios and optimise it. Here we present a framework for utilising digital twins and multi-objective optimisation algorithms. The framework consists of three parts: the digital twin, the training gym, and the communication block.

The digital twin environment will be taken from the experience repository. Digital twins themselves are a combination of three primary components: a virtual twin, a physical twin (i.e. the physical manufacturing system), and the data flow component. The data flow component is used to feed data from a physical twin to its virtual twin and returns information from the virtual twin.

The digital twin and the training gym share data via the communication block. The communication can be sockets, OPC-UA, MQTT, TCP/IP or other communication protocols depending on the application domain, and can be dynamically chosen by the reconfiguration engine depending on the system context. For instance: if a cloud-based environment is needed, then the MQTT approach could be used to enable communication.

In the training gym, different artificial intelligence algorithms and approaches are used to optimize the simulation in the digital twin to meet the different optimization targets. For example, if the optimization target is multi-objective, then multi-objective optimization algorithms are selected. If the manufacturing system looks to optimise a single parameter with the new product – such as faster operation, or lower cost – then reinforcement learning can be used in the training gym. For highly complex or difficult problems, deep reinforcement learning will be suggested by the decision engine. After the optimized result is found and evaluated successfully in the simulation environment, the optimized result (such as the optimized robot program, robot path, and the structure of the production line) will be applied to the physical device via the orchestrator. The orchestrator may make direct changes to PLC code or robot controllers to instantiate the change or make recommendations to human operators to make the changes manually. The orchestrator will receive change requests via the communication protocol suggested by the reconfiguration engine.

4 Application

This section describes a demonstrator used to verify the effectiveness of the proposed approach in achieving system reconfiguration. The following example illustrates the reconfiguration of a pick and weld production station, using Siemens Tecnomatix Process Simulate as the digital twin enabler. As shown in Fig. 2, this use case consists of two KUKA KR2700ultra robots, one conveyor for transporting the product, one welding station, and one work table for storing the parts after welding. For the two KUKA robots, one robot (robot 1) is armed with the pick and place end effector. The other robot (robot 2) is equipped with an arc welding gun.

Normally, robots are drilled into concrete or mounted to metal plates, so cannot quickly be moved. This application applies a modern reconfigurable approach that allows the robot to find the optimized position at first in the simulation environment and then applied to the real physical device. This is either done before the cell is commissioned to determine the optimal location, for situations where the robot can be moved to different locations with a reconfigurable flooring solution (which can be seen in Fig. 2), or the robot will (depending on payload restrictions) be placed on an automated ground vehicle (AGV) and moved to the new position after it receives the optimized location. The process consists of five steps:

  1. (1)

    The product (yellow part) is placed on the conveyor by the previous process in the production line.

  2. (2)

    The part will be transported via the conveyor to robot 1 for picking.

  3. (3)

    Robot 1 will pick the part up and move it to the welding station.

  4. (4)

    After the part is successfully put on the welding station, robot 2 will perform arc welding on the product.

  5. (5)

    After welding, robot 1 will pick the product up and put it on the worktable, and return to the home position.

In this use case, it is assumed that the customer wants a welded cubic product and the decision-making engine has already decided that a system layout reconfiguration is needed. Both robots need to be relocated to their optimal locations (based on the suggestions from the reconfiguration engine) in order to optimize the production time of the process. Optimizing process time is a common industrial requirement, for example, to minimize cycle time or meet a takt time. An additional constraint on the optimisation is ensuring that neither robot collides with other parts during operations.

The application for the framework of the system reconfiguration engine for this use case is shown in Fig. 3. The digital twin environment will send operation time and collision situation to the training gym via a socket (C#) based on Tecnomatix .NET API. Tecnomatix .NET API is connected to the Tecnomatix Process Simulate simulation environment with Tecnomatix .NET viewers.

In the training gym, after optimization via the selected artificial intelligence approach, the updated robot location will be sent to the digital twin, to get a new operation time and check for collisions. After all the iterations, an optimised location of the two robots will be found. The results (locations of the robots) will at first be validated in the digital twin. Then the final optimized robot locations will be sent to the orchestrator. It is the role of the orchestrator to communicate the robot locations to the relevant resource that will make the change - e.g. the human worker to arrange the robot move, or the AGV the robot is mounted on. Finally, the physical KUKA robots will receive the new optimized robot location via PLC. In this example, the CEE (Cyclic Event Evaluator) simulation mode in Tecnomatix Process Simulate is used. The CEE, which functions as a PLC, is used inside the Tecnomatix Process Simulate to control how a typical robotics simulation progresses using logic. Once the start signal is true, the simulation will start. Originally, there are no robot move relocation functions defined. With Tecnomatix .NET API, the move operation can be generated in each simulation. In the first iteration, the training gym will send random coordinates of both robots to the simulation environment, and then two object flow operations in Tecnomatix Process Simulate will be generated and be linked with other operations.

Fig. 2.
figure 2

Layout of the simulation environment

Full size image
Fig. 3.
figure 3

Framework for reconfiguration engine

Full size image

To let the result converge faster, penalties are given to situations where a robot collides with other parts or cannot complete the operation due to reachability. As a multi-objective optimization problem, multiple different artificial intelligence approaches can be applied to this use case. The reconfiguration engine can choose which approach is the best solution based on past experience. In this use case, two different types of optimization algorithms will be used to do robot location optimization. One is the global Particle Swarm Optimization (PSO) approach. The other is a Genetic Algorithm (GA). The sequence flow of these two algorithms is listed below (Fig. 4). PSO is a bio-inspired algorithm that searches for an optimal solution through iterative improvement. In this use case, the robot’s initial random locations will be set in the initialization swarm, and progressively optimized based on the simulation time received from Tecnomatix Process Simulate. GA is a search heuristic inspired by the theory of natural evolution. This algorithm reflects the process of natural selection where the fittest individuals are selected for reproduction in order to produce offspring for the next generation. The hyper-parameter for the PSO and GA are listed below (Table 2).

Fig. 4.
figure 4

Flow diagram for PSO and GA algorithms

Full size image

After the training gym has finished the optimization, the optimized location of the robots will be found. The key performance indicators in this example are the cell’s total operation time and the number of iterations the algorithm took to find an optimal operation time. The comparison of these two approaches is listed below (Fig. 5 and Fig. 6). The reconfiguration engine will choose the scenarios based on different performances. From Fig. 5 and Fig. 6, we find that the GA algorithm converges quicker than the PSO algorithm approach for this use case. Furthermore, the optimized time found by the GA algorithm is almost the same as the optimized time found by the PSO algorithm approach. The best process time found by the PSO algorithm is 23.72 s compared to 23.74 s for the GA approach. In this situation, the reconfiguration engine will recommend using the GA algorithm as the optimized approach and can save this information in the data bank for future reference.

Table 2. Hyperparameters for Genetic algorithm and PSO algorithm
Full size table

There are some limitations to the approach used in the applications. For instance, if the initial position of the robot is too far from the work-piece where they can’t execute the operations, the result will not easily converge – if the robots can’t execute the operations, the training gym will always receive a penalty (bigger than the operation time) instead of the valid operation time. Convergence speed is also highly dependent on the chosen penalty value.

Fig. 5.
figure 5

PSO approach

Full size image
Fig. 6.
figure 6

Genetic algorithm approach

Full size image

5 Conclusions and Future Work

The design and operation of manufacturing systems are increasingly changeable as the markets require quicker responses to new products, supply disruptions, and volume demands. Reconfiguration and optimisation of the production process in response to external changes is a difficult challenge for complex and flexible systems. This paper proposes a new framework to enable the system to find optimized operation parameters and configurations autonomously. With this framework, the manufacturing system reconfiguration can be enabled at first in the simulation environment and then deployed to the physical system. Though currently applied to a simulation environment, our next step is to apply the framework to our physical robotic manufacturing cells. More key performance indicators will be introduced as optimisation criteria for this framework. Lastly, more complicated use cases will be considered to use this framework. Specifically, applications where manufacturing is not limited to one workstation.