This section presents the application of the OCP and AF scenarios in the I4.0 community demonstrator myYoghurt in combination with I4.0 demonstrators at HSU and TUM.
Another contribution from the authors in (Vogel-Heuser et al., 2020), compares how agent-based systems can meet emerging challenges in those I4.0 scenarios. Software agents are an option for the realization of such OCP and AF. Due to their features, MASs are mostly well appropriate for representing their I4.0 components and enabling I4.0 interactions. Agents in different I4.0 demonstrators can recognize and integrate not only the necessary I4.0 scenario languages, but also the essential methods for self-organization and self-optimization in value creation (Vogel-Heuser et al., 2020).
The myYoghurt demonstrator is a joined academic initiative from automation research that implements I4.0 scenarios across multiple universities (Mayer et al., 2013). Within the myYoghurt scenario, an agent registers with the coordinator for a specific capability or production process, e.g. yoghurt or lid production (Mayer et al., 2013). Subsequently, the suppliers of a production step are responsible for the fulfilment of an order. Due to this separation of concerns within the limited scenario of myYoghurt, a detailed analysis if a certain product can be manufactured by a production partner is not necessary, as the registration at the DF is sufficient. The coordinator asks for and selects proposals from participants within the production network. In contrast to (Platform Industrie 4.0, 2017a), where the selection process is carried out by the regular customer, in the myYoghurt scenario the coordinator is not only responsible for obtaining suitable offers from production partners, but also for selecting the most appropriate offers for inclusion in the offer available to the customer. To reduce complexity during order creation, the customer, which is in this case, the end-user, is not supposed to decide between different manufacturers or production processes, but the CPPS itself by negotiation in between all participating plants. However, that requires every participant’s agreement to the algorithm that determines how offers are rated within the network. In this implementation, the lowest price is the essential criterion for the selection. Possible cost functions can include factors like material selection, set-up costs or location of production facilities (logistics) as well as trust. Additionally, if the participants of the coordination algorithm stem from different companies and use real prices (in contrast to transfer prices) the figure has to include the company's profit margin (Gehlhoff et al., 2019). The coordinator also implements scheduling functions that for example take sequence restrictions into account (Weyrich et al., 2014). These have a major impact on possible lead times of a product because some processes can occur in parallel (yoghurt and lid production) while others can be carried out only sequentially (yoghurt production and refinement). Communication is handled by a client-socket-based protocol, the Joghurt-Production-Protocol 2 (JPP2) (Mayer et al., 2013), which is a lightweight but inflexible XML-based communication protocol.
For the case study of the scenarios, the demonstrators serve as real plants connected by an MAS to form an agent-based network of production systems. The institutes HSU and TUM are equipped with a variety of demonstrators for research and education and are close to industrial applications. The demonstrators address the automation of different domains like logistics or process industry. To present both scenarios using an MAS, each institute integrates existing demonstrators with similar capabilities. Overall, the production of yoghurt bottles should be considered for the two scenarios. For this purpose, each institute integrates a demonstrator that produces the actual yoghurt as well as a demonstrator that produces the necessary lids for the bottles. In addition, each institute provides an intralogistics demonstrator that links the intralogistics processes with each other. Most of these demonstrators are already equipped with agent systems from previous research projects. However, as the Self-x logistic demonstrator is not yet operated by a running agent system, the integration of a new manufacturing plant into the agent automation platform is being investigated. The used range of demonstrators include the following:
myYoghurt (TUM) A modular material flow and process demonstrator for implementing and evaluating different control concepts in the field of intralogistics. The process technology section of the demonstrator consists of a (process) workstation and two filling stations for dispensing yoghurt. The demonstrator serves as a testbed for research in the field of intralogistics and process technology intending to improve the interchangeability of components, both software and hardware.
Extended Pick and Place Unit-Demonstrator (xPPU, TUM) A production demonstrator, which consists of storage, manufacturing and logistics. Because of its versatility, the PPU supports various production scenarios of the manufacturing domain, such as hardware customization.
Self-x (TUM) A modular Material Flow demonstrator for implementing and evaluating different control concepts in the field of intralogistics.
MPS500 (HSU) A flexible manufacturing system that produces customized air cylinders and thermometers.
PL (Production and Logistics demonstrator) An MAS that controls production and intralogistics processes.
MODVA (HSU) A modular process plant that can be controlled by remotely calling different recipes.
These demonstrators differ in many aspects like accessibility through different protocols, modularity or automation platform requirements. Table 3 shows the result of an analysis of these demonstrators and relevant criteria.
The demonstrators were implemented using different platforms (e.g. JADE or C#) and accessibility protocols. This drives the need for an automation platform-independent integration approach that is implemented with OPC UA.
In addition, to integrate multiple production facilities within the same production network, a unified communication platform and front-end (human interaction) are needed, which currently does not exist. Security measures are also an important factor for robustness that is currently not sufficiently addressed. OPC UA provides these functionalities and enables a unified approach to integrate the demonstrators within the same network and also provides comprehensive security measures and common standards (Vargas et al., 2017).
Summarizing, the case studies are highly related to demonstrators and standard automation platforms from market-leading automation vendors available in HSU and TUM. Still, the MAS of this work becomes an automation platform-independent by the interoperability of OPC UA, as an independent service-oriented architecture.
Order controlled production
The OCP scenario was implemented within the myYoghurt research project. It enables the automated control of the production facilities, initiated by a customer via a web front-end, at the HSU and the TUM. The MPS500, the MODVA and the PL serve in this scenario as exemplary production facilities that can produce lids (MPS500) and yoghurt (MODVA) respectively or organize intralogistics activities (PL).
The implementation is building on a legacy agent-system, in which the agents “speak” the already mentioned JPP2, including the DF-AMS, coordinator and system agents. The DF-AMS combines, in this case, the yellow-page (who provides which service) and white-page (addresses of participants) services. The coordinator implements the already mentioned order management functionalities. Keeping these agents and their ability to speak JPP2 makes it possible to connect agents that do not possess OPC UA capabilities as well. Within this agent-system, the DF-AMS agent is equipped with OPC UA capabilities to communicate with the OPC UA server and thereby serves as a gateway agent. In Fig. 4 the gateway interactions OPC UA—JPP2 are marked green and JPP2—OPC UA blue.
The higher-level system agents start the actual production by triggering a Manufacturing Execution System (MES) that controls the MPS500 and a process control system that controls the MODVA. It has been decided to keep the exiting low-level control of the production equipment to preserve real-time responsiveness (Leitão et al., 2015) and to enable the agents to start production processes by triggering the existing control components. The P&L demonstrator can provide intralogistics functions. However, these require additional interactions that are not elaborated here. In (Spindler et al., 2016) this subject is covered in greater detail.
Order management using OPC UA
One of the central components of the OPC UA information model for the OCP scenario is the “CreateGetOfferMessageObject”-method. The website’s OPC UA client uses this method to create nodes for every order that the customer has posted via the web front-end. The method input is the desired yoghurt configuration. An order node then contains every information the agent system needs to make on offer. The interactions required for the order management are depicted in Fig. 4. The agents within the legacy agent-system are requested via JPP2 to make offers for those parts of the order that match their specific capability. OPC UA enables the integration of further demonstrators without using JPP2 or adjusting the implementation of the coordinator or other legacy agents. Agents that do not speak JPP2 subscribe the getOfferMessage-Type directly and call the createOffer-method to post an offer. This offer is routed to the coordinator by using post and subscribe functions and the DF-AMS gateway agent, which translates the message to JPP2.
The agents in this demonstrator use details of the order (e.g. cost of materials and order size) to calculate their costs. Afterwards, they calculate prices based on the margin that they aim for and additional optimization that includes the conditions on the market place and possible synergies (Gehlhoff et al., 2019). The coordinator chooses suitable offers to create the total offer and determines the total price and delivery date for the order. This price, as well as the expected delivery date, are returned to the customer by setting specific variables within a separate folder for the specific offer. The OPC UA server informs the website’s OPC UA client about changes in that folder (realized with an OPC UA subscription); the client reads the new values and manages the information of the customer. Upon receiving this feedback, which can also be a refusal, e.g. if a required capability is currently not available in the production network, the customer can decide whether the order is to be produced with adjustments, postponed or cancelled. This dynamic pricing model reflects the flexible production network configuration, i.e. the nature and number of participants, which can result in very different production conditions. For example, a large number of yoghurt producers represented on the network will lead to lower prices as these suppliers will participate in price competitions. Another solution would be to use predetermined prices or price ranges displayed to the customer, for example, suggested in (Platform Industrie 4.0, 2017b). However, this is not part of the original myYoghurt getOffer configuration, which is used in this approach, and it is also less responsive regarding the resulting price that is displayed to the customer.
If the customer accepts the offer, the OPC UA client that resides at the website will call the “AcceptOfferMethod”. This method instantiates a setOrder-message that is read by the DF-AMS agent. The DF-AMS again takes care of implementing the JPP2 protocol and sends a “setOrder” message to the coordinator. The coordinator then splits the message and sends setOrder messages to every agent involved in the order to start the actual production. Agents that do not speak JPP2 are contacted by creating specific return messages in their inbox. Figures 4 and 6 also display where the agents connect to the resources they represent (start production). After receiving the setOrder-message they take the necessary actions to initiate the production process. This can happen simultaneously across an arbitrary number of production sites, as the agents run on distributed hard- and software.
In the case of the production facilities that are located at the HSU, this process is depicted in Fig. 3.
Even-though both production facilities use different software to control the production (MES and Zenon Runtime), the agents that represent them within the production network are to a large part identical. They only need to implement the machine-specific behavior or interface to connect to the machine. This results in different parameters that are related in the “start order” command and different data that has to be read (order information). The ubiquitous availability of information processing capabilities enables the agents to follow decentralized and flexible coordination schemes within their factories as well. As outlined in (Gehlhoff & Fay, 2020) they can, for example, be connected to workpieces and manage the schedules of products and resources schedule or connect to a CPS that provides information about machine availability etc. (Zhang et al., 2017). This kind of information enables the agents to adapt to new circumstances on the shop floor and thus provide a more robust and flexible manufacturing control approach. In addition, as indicated in Fig. 6 the production process is usually coupled to transportation processes that the agents can coordinate as well (see for example (Sundermeier et al., 2019). In addition to the general order management functionalities, OPC UA enables displaying the current production progress, e.g. by connecting embedded OPC UA server from production facilities to the website (marked orange in Fig. 4). Figure 4, bottom part presents a preliminary order schedule for customers for approval of price and delivery date before submitting the final order. During order creation, the folder OrderNo1001 is created by the Create Get Offer Message Object-method (Fig. 5, upper left part showing the namespace of the application). Values generated by the agent system (price, delivery date) and order status (missing final order confirmation) are always available for customers (Fig. 4, upper right part customer accept = false).
Configuration of the production network
The agent-based approach based on OPC UA eases participation of new companies offering their production facilities in the network. They can either register themselves by implementing the traditional, direct communication to the DF-AMS via JPP2 or OPC-UA. Choosing the first option: an agent representing the production facility communicates the available capabilities of the production facility to the DF-AMS agent. As a prerequisite for registration, the agent has to know the IP and port combination of the DF-AMS agent as well as the semantics and syntax of the JPP2 protocol. Choosing the second option OPC UA, the agent representing the manufacturing facility could also directly subscribe the GetOfferMsgType to receive information about new getOffer-messages on the OPC UA server (see above). Using this OPC UA approach, an agent does not have to “speak “JPP2 explicitly as the.
OPC UA server combined with the DF-AMS gateway agent can implement methods that can provide translation functionalities. It is important, however, to keep the coordinator agent in the loop as he implements the decision logic and selects, which proposals are included in the final offer that the customer receives.
However, translation capabilities can also be implemented within the coordinator agent. An abbreviated version of a communication architecture that follows this design is shown in Fig. 6.
An agent that registers via the OPC UA server must know or discover the information model that underlies the message types used on the server, the in- and outputs of the necessary methods as well as the login data that restricts the access to the server, which must be known a priori.
While the application scenario OCP focuses on the flexible operation of existing plants driven by products in intelligent networks, this scenario describes the versatility of a single factory through physical transformation.
One major aspect of I4.0 is communication across all levels. To meet the challenges of ambiguity in information exchange, a common understanding of the messages exchanged in the agent network is required. By also detailing the interactions between the agents, a rich communication workflow is achieved.
This section covers how messages between agents are exchanged using the OPC UA protocol and the message vocabulary used to make messages interpretable for agents. The formalized process description VDI 3683 is used as a means of describing the production capabilities. The allocation of a technical resource to a process operator takes place based on various characteristics (e.g. geometry of the input and output product, process-related quality characteristics or technical data of the resource). By modelling the product, process, and resource as I.40 components, they can realize a dynamic assignment at runtime through interaction.
Due to the concept of the Digital Factory and the associated necessity to electronically describe all planning data involved in the production process, a three-way division into resource, process and product data has proven itself in practice (Drath, 2010). All three views are linked with each other. With the product, process and resource model (PPR model) and the data model of the VDI/VDE 3682 guideline for formalized process description, it is possible to describe the capabilities of resources in the sense of machines and plants as well as the requirements on the production process by products. An important feature of the three different elements is that they are closely related: A product is processed by a resource within a process (Fig. 7). This three-view concept is a prime example of networked engineering data (Schleipen & Drath, 2009). For agents, it serves as a recipe for uniformly describing their abilities.
Agents responsible for executing a manufacturing process need detailed knowledge of the configuration of the production unit they control. This requires the existence of an appropriate knowledge base when commissioning the agent. Agents with primarily coordinating and monitoring activities need knowledge about other agents and their condition.
The purpose of the description model is to provide all necessary information for the classification of the production systems in a semantically uniform way. Ultimately, it should enable the agents to derive the corresponding production plan or to derive the basic suitability for the fulfilment of the inquiry, starting from an inquiry for the manufacture of a product.
Adaptions by agents
In adaptable manufacturing systems, several products can be manufactured using the same resources. If possible, the factory setting is reused for the different products and only the action sequences are updated accordingly. In some cases, the factory setting may need to be changed. Changes include adding or removing resources or changing the location of a resource within the factory. It is also possible to use different modules of a resource for different products. In order to generate action sequences for different products automatically, two aspects must be considered during generation: the product description and factory setting. By decoupling the software control from the machines, the network of agents forms an adaptable architecture that can be flexibly scaled regarding the production system’s needs. Coordinating high amounts of individual and distributed modules with specific interfaces is a challenging task that agents can tackle by providing fast communication channels to a multitude of automation platforms and establishing networks of interconnected facilities.
Modularization of the software in the form of agents with the capability to distribute functionalities makes it possible to scale the network of production facilities. To meet the flexibility requirements of hardware modules, the software architecture in an adaptable factory must also be modular. By linking agents together, messages or updates can also be automatically sent to all using the shared communication network. In order to be able to interact with other systems that may have a different implementation, a communication channel via the OPC UA protocol is selected to unify the exchange of information. In addition to the communication channel, the type of messages used is also displayed uniformly. The capabilities of the individual production sites are described by a uniform vocabulary so that all different MASs can understand the messages and express themselves by using the same vocabulary. The Configuration is updated once the agent detects a change in the setup.
An integration agent is responsible for managing the network of production facilities. By linking different facilities, it creates a distributed control system consisting of one distributed facility. These systems, in particular logistics systems, consist of various numbers of modules and can be flexibly combined to produce a new type of product or offer new functionalities. Within a network of different agents, individual configurations can be distributed very quickly. Agents can also aggregate different functions due to their modular character. In this way, functions can also be dynamically shifted between agent systems and thus react quickly to changes in the plant layout. By using agents, the network can distribute software modules for controlling the manufacturing processes within the MAS.
Coupled by agents that can all exchange information via a uniform communication channel, the demonstrators in Table 3 form a network that enables adaptation across different granularity tiers. Each demonstrator has a counterpart at the other institute, such as for yoghurt or lid manufacturing. Using OPC UA, the agents can communicate even if their local system primarily uses a different protocol (e.g. JPP2). Within this network, agents can react flexibly to any changes in their plant and thus relocate functionalities to other facilities by negotiation with other agents or request services from them. Using an OPC UA interface, the different MASs can also exchange information with each other, even if their implementation differs. Agents can also react locally to changes in their plant. If a filling station fails at the myYoghurt system, the agent system adjusts the routing of the workpieces so that the failed unit does not result in the system shutting down.
As described in "MAS communication infrastructure using OPC UA" section each institute has three facilities involved in the production of yoghurt bottles (cp. Table 3). Except for the Self-x demonstrator, each system is operated by an individual agent system. In order to integrate Self-x with its logistics services into the network, an independent MAS that can manage the control processes of the plant was initially set up. For the integration OPC UA was used, through which a dedicated agent in the Self-x MAS contacts the network and registers himself with his capabilities. After registration, the system with all its functions is at the disposal of the network. Therefore, this process of establishing contact and capability registration can be considered as an agent-based plug & produce process.
In addition to adopting new plants, the agent network can also be used to shift assignments of facility configurations when a change occurs (e.g. as a result of a failure of a component). The agent network can be understood as a collective unit of which each demonstrator is a part of. Figure 8 shows an example of the physical distribution of the plants and how the production configuration can be shifted between the institutes as a result of a change of a component’s state. The Fig. 9 illustrates the adaption of configurations of the agents. For example, the transfer of the ability to produce yoghurt from the myYoghurt plant, which is necessary due to the failure of one of the two filling stations, can be either directly adopted locally to its remaining filling station or even relocated to the MODVA plant.
Likewise, the ability to manufacture lids can be shifted from the MPS500 agent to the xPPU agent in the event of an adaptation. In Fig. 9, the adaption process is illustrated. The myYoghurt, xPPU and MPS500 are each controlled via the agent architecture presented. Both systems are registered with their respective capabilities (PPR) in a central directory, the DF. A Coordination Agent accepts new orders and coordinates them by delegating the production of individual sub-products to the appropriate production facilities by comparing the order with the capabilities registered in the DF. When an order for a yoghurt arrives, a new production order is created and forwarded to the agent system. Based on the product characteristics and the capabilities stored in the DF, the myYoghurt system is selected as a suitable production system and receives the production order.
Within the myYoghurt system, the order is assigned to a new product agent who coordinates the production, as introduced by (Kovalenko et al., 2019). Whenever new lids are required to seal the filled yoghurt bottles, a production order for lids is sent to the xPPU. If the xPPU reports that no lids are currently available, it reports an error back to the myYoghurt system. This sends a response to the Coordination Agent. The Coordination Agent in return compares the stored production properties in the DF with the requirements for lid production and selects the MPS500 as a suitable alternative. The order is now forwarded to the MPS500 with the contact information of the myYoghurt system. The agents of the MPS500 then start producing lids and report to the agents in the MyYoghurt system.
To always be able to provide information on the current status of their hardware components, the agents need to constantly follow the modifications to the hardware they represent and reconfigure themselves according to the changes. The management agent takes over the device management of the network by keeping track of the respective systems and their configuration. This makes it possible to automatically reconfigure the configuration after the conversion and to initiate the redistribution of production services.
The software architecture of the agent system responsible for handling the logistics operations within the myYoghurt plant is depicted in Fig. 10. The Software on each of the Programmable Logic Controllers (PLC) contains two parts: a classic control program for controlling the hardware (controlling the actuators, reading the sensors) and an agent system (transport agent and framework). The active framework functions as the central intelligence (coordinator) of the entire plant’s agent system, which simultaneously serves as the uniform interface to higher-level systems (e.g. Warehouse Management System, ERP, Databases, etc.).
If the myYoghurt’s conveyor system is modified or converted because a new product is required. The system and software-related changes must be identified and automatically transferred to all participating agents. When the current layout is changed, two cases are distinguished: Adding and removing a conveyor belt. When adding an additional conveyor belt to the system, it must be registered in the MAS and its functionality must be published. The registration process is completed in a few minutes and is performed by the active coordinator. In order to determine the current plant layout, the coordinator needs to require the description of the newly added conveyor, which it requests from the designated transport agent.
Another advantage of MASs in this context is that additional services can be integrated with ease into an existing MAS. Furthermore, the abilities of agents to make decisions autonomously and to learn from past events lead to faster reactions and optimized control. The joint research demonstrator myYoghurt embodies such a distributed production facility that is powered by agent-based control applications. Each production facility resides in their very own location and communicates with other remote systems by the means of a cloud-based agent management system that coordinates all registered resources within the network.
Due to its application-independent nature, the developed MAS is open for extensions. Additional arbitrary systems can be integrated with ease. In addition, agents can introduce new services or products and advertise them within the network by the means of their underlying information model.
Integrated I4.0 scenarios
The integration of different MASs within a unified architecture enables the System of Systems (SoS) concept (Colombo et al., 2013), by joining its previously disjunctive components. MAS technologies take advantage of agent-inherent characteristics to develop large and complex SoS that exhibit modularity, adaptation, reconfiguration, and responsiveness to condition changes (Karnouskos et al., 2019; Leitão et al., 2016).
Such a use case is, for example, the exploration of possible paths through the system that a product can utilize. The result of this kind of search is commonly known as the process plan. To determine possible operation or process sequences, a coordinator agent (or any other entity responsible for this task) contacts each RA with a request that contains the description of the required in and outputs to produce a specific product or provide a particular aspect of its production. This description, for example formulated in the formalized process description (Gehlhoff et al., 2020), can be matched against the RA’s capabilities. In case the RA can provide the necessary capability, it answers accordingly. Otherwise, it can relate the message to further RA’s that can, in turn, check if they can contribute to the requested task. Note that it is advisable to formulate the requests as ‘feature-neutral’ as possible, i.e. by stating that a whole is required instead of specifying that a drilling process must be provided. This enlarges the search space and can improve the solution. A product agent (or the coordinator agent) that is equipped with the resulting flexible process plan can utilize this plan during the scheduling process (Gehlhoff & Fay, 2020). It can evaluate different possibilities to be manufactured, i.e., calculate alternative paths through a graph. Besides, it can use the latter during the execution process if the preferred route is blocked.
There is also the possibility to provide further functionalities during the checking of producibility. If the capability model of the RA’s is extended by, for example, an algorithm that provides the means to check adaptation options (Hoang et al., 2019). This enables the MAS to not only check the producibility with regards to the current and alternative configurations of the system, i.e. the selection of a specific tool in a multi-tool machine. It also provides a means to integrate possible adaptations, like for example the enlargement of a gripper to handle larger workpieces, into the solution space.