The I4.0 focuses on key aspects of smart manufacturing that can be explained as interactions between the following features [47, 60]: i) horizontal and vertical integration through value networks and within a factory or production shop; ii) life cycle management that refers the end-to-end engineering; iii) the human beings coordinating the stream value; and iv) the security to achieve the confidentiality, integrity, and availability of assuring data (transfer and storage). Likewise, the mass personalization known as the Additive Manufacturing can combine the smart manufacturing to a paradigm move for the I4.0 . In order to facilitate and promote the smart manufacturing aspects mentioned above, RAMI 4.0 provides a flexible architecture based on functions and information levels within 3D dimensions. As illustrated in Fig. 7, there are different applicable standards , to follow the guidelines in the RAMI 4.0 model.
The RAMI 4.0 model provides a structured view of the multiple levels (even a specific Asset level) using an architecture consisting of three axes (see Fig. 7). The aim of the model is to create manageable segments (sub-models) by combining the different axes at each point in the asset’s phases, to represent each relevant characteristic. The following items describe the RAMI 4.0 axes distribution [47, 60]:
The first axis is named the “Architecture hierarchy”. It is based on the traditional IEC 62264-1 (ISA 95) and IEC 61512-1 (ISA 88) standards and their levels’ hierarchies. The goal of this first axis is to define assets and their combinations with the necessary precision, since the description of RAMI 4.0 is a purely logical one.
The second axis is named the “Layers.” This one uses six layers to represent the relevant information for the multiple assets’ roles: Business, Functional, Information, Communication, Integration and Asset.
The third axis is named the “Product life-cycle.” Based on IEC 62890, it represents the lifetime of an asset and the value-added process.
In the next section, the authors of this paper address the alignment of sub-agent patterns with the RAMI 4.0 model by comparing only two dimensions. Many similarities can be found between agent-based architecture for CPPS and the RAMI 4.0; however, the Product Life-cycle axis is out of this paper’s scope.
MAS architecture based on RAMI 4.0 model
Regarding the Layers axis, the MAS proposed based on patterns should describe the I4.0 components in terms of properties, system structures, specific data and functions, and their external behavior. Since the present layers do not conform to the ISO-OSI guidelines, it is not mandatory for a RAMI 4.0 layers to provide the corresponding information. As a result, some layers can also be ignored in specific domain systems that are not applicable. A layer just characterizes parts of asset’s behaviors and their connection between adjacent layers. A possible definition for the agent-based CPPS architecture is proposed in the upcoming paragraphs.
An Asset is a physical/logical item having actual value to the organization [34, 60] (e.g., products, equipment, software, human resource, standards, and documentation). In case of a physical asset, according to the IEC TS 62443-1-1, for industrial control automation, the device under control can contain the largest directly quantifiable value. The Asset Layer is the lowest level of the RAMI 4.0 model because it reflects the physical components, administrated by other upper layers, which are in the cyber world.
The Integration Layer is a type of adaptation for the transition among physical and cyber worlds. The main aim of this layer is to convert a physical variable into a digital one. The resulting data is converted according to specific formats. Therefore, the Resource Agent is the most significant entity in this layer. For example, RAs have the adequate subroutines to send and receive data for a controller to regulate the speed in a conveyor (sub function). Additionally, operator interactions could take place in this layer, e.g., via Human Machine Interfaces (HMIs).
The Communication Layer covers connection lines according to the guidelines of I4.0. This layer distributes information to other I4.0 components and receives data back from them. The base of the Communication Layer absolutely follows the seventh ISO/OSI layer’s guidelines . A Communication Agent, using a uniform data presentation contained by the CPPS, can standardize the communication methods. In addition, CAs provide services for control in the route of the adjacent Information Layer. The definition of communication technologies within I4.0, such as Machine-to-Machine (M2M) and Machine-to-Business (M2B), e.g., the OPC UA (IEC 62541), clearly applies to direct communication . Other communication protocols, such as FIPA specifications or MQTT, are much more flexible; therefore, they enable indirect communication [31, 34]. For this layer, the patterns AMS, DF, and MTS from FIPA can support the Communication Layer and will be further explained in the Functional Layer. For an agent-based CPPS architecture, the scalability and the interoperability of industrial communication networks (IEC 61784) are decisive for numerous smart components and functions as well (e.g., RFID sensors).
The Information Layer defines the information for significant functions and data storage sites of a particular asset (i.e., the Cloud) . The PA could be a logic abstraction for a product in this layer. Since the PA holds its own information of procedures and plans, it is responsible for coordinating its own production. The PA pattern is a special entity of the agent-based CPPS, which orchestrates the execution of processes steps (with cooperative skills to interact with RAs, CAs, and AMSs sub-agents). The Information Layer is important to understand the different partial models of all the sub-agents, including existing data exchange formats for each specific case. To fulfill the contents of this layer, its 4.0 implementation should be based on models integrating different fields with reliable and standard methods. Here, RAMI 4.0 suggests Automation Markup Language (AutomationML) specifications, which trace a modular document structure with the aim to join the diverse and modern engineering tools in their heterogeneous disciplines (e.g., mechanical engineering and electrical designs). Thus, for information models, the AutomationML can enhance or adapt the existing XML-based data formats, integrating AML and OPC (DIN SPEC 16592), or using IEC 62424 (CAEX topology), ISO PAS 17506 (Collada), IEC 61131 (PLC open XML). It is also possible to use engineering and designing tools, e.g., ISO/IEC 19514 (UML/SysML) . For the semantics of AutomationML, the standard’s properties from ISO 13584-42/IEC 61360 (eCl@ss: classification and product description) with the Common Data Dictionary (IEC 61360 CDD) can be applied. All of the above comply with the standard for the Digital Factory (IEC 62832).
The Functional Layer follows the rules of I4.0 by assigning all logical functions and services of the assets. These technical functionalities get data from the Information Layer and deposit them back in the same layer, as methods and decision-making logic (e.g., mathematical functions) [34, 47]. According to the use case, these methods can also be executed in other lower layers such as the Integration, Communication Layer or Asset Layer. Therefore, AMSs have a fundamental role in this layer, since it contains methods for registering and deregistering modules to and from the system. Other leading patterns are the knowledge base modules because they allow the RAs or PAs to check whether the parameters of the technical systems and processes are kept within the predetermined functional limits.
The Business Layer is the highest level that defines the pertinent business procedures with their structure requirements and the business-related features of the assets (e.g., regulations, legislative requests, contracting, and licensing) . This layer does not refer to any concrete systems such as the ERP, since the Functional Layer (in the factory plant context) usually sets the ERP’s functions. Since no pattern was found that could fulfill the Business Layer’s requirements, this layer will not be further researched in this paper.
Each sub-agent located in the agent-based CPPS architecture, as Fig. 8 shows, is not required to have a fixed location in the RAMI 4.0 layers.
Given the model associates multiple layers in two dimensions, each MAS component pattern is a primary part with a specific role in its respective layer, but they are possibly joined with the other adjacent layers, as described above.
There is another important second axis from RAMI 4.0, the Architecture hierarchy that represents the hierarchy position of functionalities and responsibilities within the factories/plants. This functional hierarchy is not only the equipment classes or the automation levels of the classical pyramid. As mentioned above, this axis follows the ISA 95 and ISA 88 standards to realize the classification within the plant (see Fig. 8) . However, RAM I4.0 considers other levels to cover as many areas as possible from traditional industry to the new factory automation. New terms based on ISA 95 levels (see Section 5.6) are established such as the Enterprise Level (L4), Work Unit Level (L3), Station Level (L2), and Control Device Level (L1). I4.0 considers other multiple equipment or systems within the factory because not only the controllers are decisive for this one. Therefore, the Field Device Level (L0) has been added below the Control Device Level, and it is a practical level of a smart field device (e.g., an MAS RFID intelligent sensor ). Moreover, not only the plant and its equipment are essential in I4.0, but also the products to be factory-made itself. Then, RAMI 4.0 adds the Product Level as the lowest level that allows standardized consideration of the product to be mass-produced and the manufacturing capability (with their relationships).
An addition has also been made at the highest end of the Architecture hierarchy axis. The two ISA/IEC standards cited only define the levels within a plant (see Section 5.6). However, I4.0 goes further by describing group corporations, interdependencies, and net of factories (e.g., alliance with outer engineering companies and component suppliers/customers). Consequently, the Connected World Level has been added to observe above and outside the Enterprise Level. Regarding the heterarchy sub-gent’s patterns in this axis, their locations are based on the first (Lo) to last (L4), as ISA 95 automation levels mention. Consequently, the Connected World and the Product levels are out of the agent-based CPPS architecture proposed in this paper.
Design pattern for the administration shell
For this section, from [1, 34], there are specific terms as important definitions for I4.0:
The Industry 4.0 component (henceforth, I4.0 component) is a “globally uniquely identifiable participant with communication capability consisting of administration shell and asset within an I4.0 system which there offers services with defined QoS (Quality of Service) characteristics.”
The administration shell is the “virtual digital and active representation of an I4.0 component in the I4.0 system and contains the manifest and the component manager.”
The Manifest is an “externally accessible defined set of meta-information, which provides information about the functional and non-functional properties of the I4.0 component.”
The component manager is “the organizer of self-management and of access to the resources of the I4.0 component…”
Looking to fulfill the requirements for the CPPS and the RAMI 4.0 (see Section 2.1), a general organization for the administration shell based on the MAS can be developed. For this, the I4.0 components for the CPPS proposed display an abstract form that defines real objects. For example, these could be a valve as a control element, a pipe as the controlled process, a sensor as a measuring element, and a PLC’s algorithm as a controller, etc. MAS architecture should be based on design patterns described above, in both ways, physical (assets) to the cyber (digital data). Moreover, Information and Communication Technology (ICT) needs to increase additionally regarding the appropriate smart manufacturing aspects such as the horizontal/vertical integrations, product life-cycle, human’s interaction and others, as mentioned at the beginning of this chapter. As shown in Fig. 9, the administration shell contains the “Header” and the “Body” parts. Both in order to provide better identification via asset(s) designations [34, 47, 60].
The intention of this section is to describe a general implementation of I4.0 components using a sub model design with the identified MAS patterns. A basic application of an I4.0 component is based on the suggested international standard of AutomationML, as a method for the Information Layer, and OPC UA for the Communication Layer. Both together could realize the Body of the administration shell of the I4.0 components with an agent-based CPPS architecture. For the Header part, the CPPS provides an adequate unique identification of the I4.0 component by a server. Also, the data representation and its function access should be integrated. According to RAMI 4.0 suggestions [47, 60], a unique identification of the object could be using a UUID (Universally Unique Identifier) or URIs (Unique Resource Identifiers, e.g., for RDF). The AutomationML concept specifies every object with a UUID that could be kept as long as the object exists. For communication, the I4.0 component provides access to technical functions pre-realized in the AMS sub-agent (with their respective DFs and MTSs), in order to enable the access to the representation of any asset’s information.
The Body part of the administration shell contains structured sub-models which might denote information and functions . A standardized format eCl@ss, which is based on IEC 61360, is suggested to describe the data and functions in a diverse and harmonize format. The features of all sub-models, in consequence will always develop a comprehensible table of contents (each I4.0 component and its respective associated manifest and administration shell). As a prerequisite for required semantics, the Header shall individually recognize administration shells, Assets, Sub models and their properties globally.
This paper’s approach assumes that a physical asset type of interest is controlled by open controller architecture (e.g., PLC) that implements lower level programing codes (e.g., IEC 61131-3). As the MAS design patterns are shown (see Section 5), the sub-agents of the different assets type can be located on all ISA 95 automation levels. Hence, for the operation of an I4.0 component, it has to be clearly specified, which technical functions are provided by the component and their configurations limits. For the PLC implementation example, adequate variables for the code should be accessible via functions with multiple OPC UA servers interlinked and following a service-oriented architecture (SoA), proposed in . As a result, the administration shell of a I4.0 component consists of multiple sub models (first is the MAS architecture) and a nonempty set of interlinked designs (e.g., AutomationML projects, mechanical computer-aided designs or CADs, interconnecting FBs/POUs models, UML classes diagrams, and others ).
Other standards can be applicable to MAS patterns and aligned to RAMI 4.0 with multiple aims (see Fig. 7): The VDMA 24582 (condition monitoring) for maintenance purposes into the Asset Layer and Integration Layer. The ISO/IEC 27000 for the security of management systems. The IEC 62443 used for the network system security and IEC 62351 for secure authentication . The IEC 61511 applies the functional safety and the IEC 62061/ISO13849 relates the machinery safety. Software quality can be valued by ISO/IEC 25010 and the ISO/IEC 25023 (SQuaRE method) . Semantic web stack can follow the W3C consortium definitions such as SPARQL, RIF/SRWL, RDF/S, and OWL. Energy efficiency can refer to the ISO/IEC 20140. Finally, configuration and programing typical tools are based on C++ plugins for control languages (e.g., mostly in IEC 61131-3 or IEC 61499 [13, 14], IEC 61804 (FBs for process control or electronics), and the IEC 62453 (Field device tool, FDT).
Discussion of the CPPS and RAMI 4.0 requirements evaluation
The majority of the requirements specified in Section 2.1 are already partly completed by the design pattern of the proposed MAS architecture. First, for the CPPS requirements (Req1.1-Req1.5), the compatibility to different applications (Req1.1) is warranted by the open software MAS architecture (see Section 5). Level independence (Req1.2) and platform independence (Req1.3) are partly achieved by applying four types of sub-agents: the RAs, PAs, CAs, and AMSs (see Section 5.6). Using the TCP/IP as fundamental communication protocols, (e.g., OPC UA) can solve parts of handling and recovery errors (Req1.4), and allows the CPPSs networks to be accessed by other applications. By distributing organizational sub-agents in the cloud (e.g., the PAs, and AMs), the agent-based CPPS is decentralized (Req1.5), as shown in Section 5. However, the multiple platform acceptation (Req1.3) and the reconfiguration of sub-agents (Req1.4) should be further examined by quantitative experimentations. As a first assessment of the platforms suitability, some experiments with these CPPS requirements were measured into multiple platforms in .
Regarding the RAMI 4.0 requirements, the proposed I4.0 components support multiple engineering disciplines and norms (Req2.1). For example, the MAS architecture is focus on the software components (sub-agents’ patterns); it is also possible to associate the physical connections of assets via CAD diagrams, according to functional considerations of AutomationML, as Section 6 mentioned. The MAS systems boundaries (Req2.2) and nestability (Req2.3) principles for the I4.0 components are aligned by the MAS’s organization in the axis of layers and Architecture axis, respectively (see Section 6.1). The general administration shell model of Section 6.2 partially gets the virtual representation of I40 components (Req2.4) and its functional properties (Req2.5), as shown in Section 6.2. However, the agent-based CPPS architecture did not specify non-functional requirements (Req2.5) yet, such as precise quality characteristics (non-functional requirements) or evaluation metrics attributes (e.g., degree to which the sub-agents cover all their tasks and objectives). The summary of the hypotheses evaluation according to the fourth research questions (RH1.1-RH1.4) is shown in Table 15.
From the eight hypotheses (see Table 1), five are true, and three are partially true, considering the evaluation of this manuscript’s authors, and the FA 5.15 expert discussions. These results are extended by fulfilling preliminary requirements (see Section 2.1) and represent the related sections of this manuscript, as shown in Table 15.
Comparison of the agent-based CPPS architecture to other approaches
Considering the two essential architecture types from Trentesaux  (hierarchical and heterarchical interaction entities), CPPS can be described through different designs with advantages and disadvantages of the distributing control decisions (see Section 2). Explicitly, hierarchy could be seen as a type of “vertical control distribution” while heterarchy is a type of “horizontal control distribution” . The type of architecture will define the quality characteristics of the production system. Traditional approaches are included into the Class 0 and Class I types of architectures, respectively centralized and fully hierarchical. What is common in these two architecture types is that they both have a main decision node, where the planning and information processing are concentrated . These classes show better optimization qualities, but a slow response and low tolerance to faults and expansibility . Thus, it is possible to construct a CPPS architecture typology that is inspired by Computer Integrating Manufacturing or CIM (e.g., ). The CPPS of Class 0 and Class I (one-level heterarchy) are applicable for CIM, since these are based on pure hierarchical interactions (e.g., ). On the opposite side, Class III uses full heterarchical interactions to lead mostly distributed architectures (e.g., ).
Class II CPPS architectures, being semi-heterarchical, can be positioned in between because they can integrate both hierarchical or heterarchical interactions (e.g., )—assimilating both advantages and certain disadvantages. The main advantages of the hierarchical type are the robustness, predictability, and efficiency. Then, in the CPPS approaches of Class II, local decisions are made taking into account global criteria and these are distributed to different controllers. Despite their advantages, traditional methods do not show the capability of adaptation due to the rigidity of the control architecture that as a result weakly responds to changes. Such types of production systems will not show the capabilities of responsiveness, flexibility, and reconfigurability . Therefore, an advantage of the proposed agent-based CPPS architecture can be found according to the classification of . The proposed architecture is classified as Class II type, since sub-agent interconnection is not strongly associated (not Class III), while there is at least a strong sub-agent connection (not Class I) . For example, a unique RAs’ network (Class III) of the CPPS could be a Class II control system with supervisory level sub-agents (with PA or AMS). Other CPPS approaches [22, 64, 65] can also support advantages of Class II as well as MAS, bionic, bio-inspired (e.g., ), and holonic. Among the last named CPPS approaches, the PROSA  and ADACOR  are the most relevant architectures. In principle, each holon shall represent a logical unit of the manufacturing system, while the sub-agent patterns could help its actual implementation [8, 65].
In heterarchical control systems (Classes II or III), long-term optimization could be hard to get and to validate, while with traditional classes (Classes 0 or I), short-term optimization is easier to obtain . In the Class III type, as long as all the entities (e.g., agents) get the equal level of autonomy, an adequate level of performance can be attained, but there is no global view of the system . As these features are disadvantages of Class III—even for MAS approaches of Class II—the proposed agent-based CPPS architecture cannot claim to be exempt from this problem and only an adequate AMS’s global response to it could address the issue.
Table 16 summarizes different CPPS approaches examples which allocate control decisions from centralized control systems (Class 0 and Class I) aiming to design non-centralized control systems (Class II and Class III). This table compares components of different CPPS approaches with patterns of this paper’s proposed CPPS architecture.
Table 17 compares advantages and disadvantages of hierarchy (Classes 0/I) and heterarchy (Classes II/III) of CPPS approaches (based on [22, 64, 65]).
Use case evaluation with an I4.0 demonstrator
The “Robot Integrated Agent Network” (RIAN) completes the evaluation of the proposed agent-based CPPS architecture. The RIAN demonstrator was presented at Automatica fair in an industrial environment . The purpose of the demonstrator was to crosslink heterogeneous production equipment and robots in a network for common customized production. RIAN was the result of a collaboration of the academy (Technical University of Munich “TUM” and Brandenburg Technical University of Cottbus “BTU”) with different industry partners , which applies the reference architecture of the MyJoghurt I4.0 demonstrator . With RIAN I4.0 demonstrator, the users could customize the bottle opener (with freely definable lettering) online and choose a delivery time depending on available production capacity (IIoT-HMI interface). A chain of production stations composed of autonomous and operator-controlled mobile transport robots defines RAIN. These stations cover the production line for the individualized bottle opener consisting of the following: cutting by laser simulation (CPPS A), injection molding (CPPS B), engraving-laser (CPPS C), packaging (CPPS D), and customer delivery by a Mobile transport robot (CPPS E), depicted in Fig. 10 (adapted from ).
Starting from the warehouse, mobile robots transport pieces between stations of the production process. Intermediate robot stations have hardware interfaces, which ensure the exact positioning of pieces or their detection by vision sensors. All CPPS communicate via an agent-based CPPS network in order to exchange necessary processing steps as well as clearances for manipulation. The current production progress is traceable for the client, for the maintenance and operating personnel. This is possible due to the aggregated reports of the individual sub-agent patterns (RAs, PAs, CAs, and AMS) into an external server (at TUM Garching near Munich), as shown in Fig. 10.
All CPPS A-E include RAs with goal-orientation algorithms (even with artificial intelligent) to achieve PAs orders. Inside the Mobile transport robots (CPPS E) and the Packing station (CPPS D) exists a hardware MAS interface (MAS ITF) and agents (CAs) which ensure by computer vision systems the exact positioning of the products (PAs) in the plant (see Fig. 10). An agent (PA) assigns initial orders to the transport robot (RA) from the storage and links the information about the process steps and the corresponding features of the products. RIAN defines a distribution of production phases for multiple participating companies and technologies (Req1.1-Req1.3).
All CPPS interactions are connected to the local hardware and accept new orders (PAfinal) after registering at the directory services (from AMS, MTS, and DF). Since the customers’ orders for products need to be decomposed into multiple different manufacturing tasks, to which the facility agents can respond, various CPPS aligned with the proposed architecture were implemented for this purpose (see Fig. 11).
A key benefit of the agent-based CPPS approach in the context of I4.0 is the linkage of heterogeneous controls. RIAN enables suitable controls to cooperate with adequate operating systems of various robot vendors (e.g., Raspberry Pi with Raspbian-Linux, Reiss robotics (now KUKA), robot controller with VX Works, FANUC robot controller with FANUC OS).
Both the implementation on suitable controls as well as on external computers is possible by using manufacturer provided interfaces. These interfaces enable data exchange between agents (CAs) and controls on the field level (RAs). Thus, the cost of changes in the software on the proprietary controls is minimized.
The configuration, changes, and adaptation of the control software (reconfigurability and reusability) are manageable by calling functions according to each manufacturer specification (CPPS) and adapting parameters or variables at runtime (Req1.4). An agent (AMS) retrieves status information from all controllers containing the state of the plant and the processing progress. Based on this information, it decides the strategy of a production unit (Req1.5). Besides the agent (PAs) extensive knowledge about the process data, there exists an encapsulation to the overall network information (AMS). Over LAN, Wi-Fi, or mobile data connections, the current production time and the price of the service are provided for all participating agents (RAs, PAs, CAs, and the AMS), as shown in Fig. 10. An Internet server is required for linking various transport and production units via the Internet. The server (in addition to the infrastructural facilities) also creates agents instances (e.g., PAs) accessible on the cloud. Therefore, different hardware platforms, e.g., PC and PLC, are connected via the Internet. The open protocol of the MAS platform (based on TCP/IP) enables connections to other implementations (e.g., based on C++).
In Fig. 12, the agent-based CPPS architecture is shown considering the Functional and Communication Layers (regarding RAMI4.0) of RIAN. The proposed architecture was used to implement a distributed production environment on Automatica fair  as a collaboration of multiple companies (e.g., Martin Engineering, Schunk, Beckhoff, and others) in different exhibition halls (A4, A5, and B5). By using the proposed MAS approach, companies were able to realize the communication, design, and application with a specific implementation of different hardware and software platforms. Each hall represents an administration shell of an I4.0 component (CPPS A-E). In the industrial context, each hall could be represented by different worldwide plants, which collaborate in a unique production process (see Fig. 12).
The industrial partner of RIAN confirmed that they were amazed by the effectiveness and ease of the collaboration . They implemented all necessary functionality needed to manage the production steps at the different facilities in the exhibition halls. The RIAN implementation required less than 3 months with less of four to six developers per academic and industrial partner.