Decision Support for the Optimization of Continuous Processes using Digital Shadows

Abstract

Decision support systems can provide real-time process information and correlations, which in turn assists process experts in making decisions and thus further increase productivity. This also applies to well-established and already highly automated processes in continuous production employed in various industrial sectors. Continuous production refers to processes in which the produced material, either fluid or solid form, is continuously in motion and processed. As a result, the process can usually not be stopped. It is only possible to influence the running process. However, the highly nonlinear interactions between process parameters and product quality are not always known in their entirety which led to inferior product quality in terms of mechanical properties and surface quality. This requires accurate representations of the processes and the products in real-time, so-called digital shadows.

Therefore, this contribution shows the necessary steps to provide a digital shadow based on numerical, physical models and process data and to couple the digital shadow with data analysis and machine learning to enable automatic decision support. This is exemplified at various stages throughout two different process chains with continuous processes: first, by using a thermoplastic production process called profile extrusion, and second, on the example of a metal processing process chain, from which three processes are described in more detail, namely, hot rolling, tempering, and fine blanking. Finally, the presented approaches and results are summarized.

1 Introduction

This contribution describes the prerequisites for the development of decision support in real time for further process optimization of continuous processes. In continuous processes, the material, either fluid or solid form, is continuously in motion and processed. This production type has been applied for nearly a century in almost all industrial sectors of production, for example, in the plastic or metal processing industry. This long history has led to well-established and often highly automated processes.

However, the physical interactions between process parameters and material behavior that determine product quality are still not well understood or even unknown in detail, especially in complex manufacturing contexts, as the interactions are highly nonlinear. For instance, the start-up and shutdown phase of processes or material property fluctuations can still lead to inferior product quality in terms of mechanical properties and surface quality. Therefore, the challenge is to predict and control final product quality within each production step and along process chains. Current approaches are based on heuristics, expert knowledge, or long trials which are usually time-consuming and costly and often do not lead to generalizable insights.

Therefore, concepts and methods presented in the context of the Internet of Production (IoP) (Pennekamp et al. 2019) such as digital shadows are combined with different algorithms to enable decision support systems to increase the productivity. Digital shadows are situation-specific real-time representations of the material behavior and the process. These digital shadows consist of simulation results from (reduced) numerical and physical models as well as of aggregated data and process knowledge. Combined with algorithms for data processing and analysis, such digital shadows can provide automatically analyzed correlations between process parameters, material behavior, and final product quality. These correlations, together with the numerical and physical models, can then be used to provide real-time suggestions for optimizing processes or adapting it if deviations occur and thus enable decision support. To demonstrate that this approach is suitable for a wide range of different continuous processes, the digital shadows and algorithms for optimization are exemplified at various stages throughout two different process chains.

The first process chain consists of only one process, namely, the extrusion of plastic profiles. In profile extrusion, raw thermoplastics are melted and homogenized using a rotating screw and formed into a continuous profile using an extrusion die.

The second process chain describes typical metal processing from which three processes are selected as examples: hot rolling, tempering, and fine blanking. In hot rolling, the material is heated up and then the thickness of metal sheets or coils is reduced by passing through one or more pairs of rolls. After rolling, the material is heated up to a defined temperature for a certain period. This heat treatment process, called tempering, aims to reduce the hardness of the material. Depending on the final product, a wide variety of sheet metal working processes such as fine blanking can be applied. In fine blanking, the material, either sheet or coil, is punched using a punch and counterpunch to produce components with high sheared surface quality.

Figure 1 shows the two process chains and the general structure of the decision support systems. The two process chains are each described separately. After a brief introduction to each process respectively process chain, the prerequisites for digital shadows are presented including numerical and physical models and data acquisition. Finally, concrete examples show the potential of Machine Learning (ML), more precise Reinforcement Learning (RL), in the development of decision support systems.

Fig. 1

Overview of the continuous processes (top) and the proposed structure of the decision support system (bottom)

Finally, the most pertinent results of the different applications are summarized and a brief outlook on the next steps is given.

2 Single Process for Plastics: Profile Extrusion

In the plastic profile extrusion chain, plastic pellets are continuously processed to plastic profiles of fixed cross-sectional shape, (see Fig. 2). In a first step, the plastic pellets are melted and homogenized inside the single-screw extruder. The melt is then shaped into the specified profile geometry within the so-called extrusion die. Subsequent to the extrusion die, the so-called extrudate exits the die, where it is still too hot to hold its shape by itself. Thus, the profile is cooled down so that the material solidifies and is fixed in shape. This process step is referred to as the calibration and cooling stage.

Fig. 2

Process sections in a classic extrusion process (Hilger and Hosters 2022)

Extruded profiles need to show minimal warpage and predictable shrinking behavior to meet market requirements with regard to geometrical, mechanical, and optical properties. The materials typically modeled in profile extrusion are thermoplastic melts. Due to their complex molecular structure, these melts are sensitive to temperatures and shear rates, but often depend additionally on past deformations. This makes it not only difficult to design and set up the process but also challenging to numerically model these materials. The die design, for example, is still largely empirical and based on experience and manual labor.

In the context of this contribution, the die design and the calibration unit will be discussed in more detail. In the design process of extrusion dies, the complex boundary conditions and material parameters should already be taken into account in the numerical design, so that, for example, by providing a mostly uniform melt distribution, the deviation between the target and extruded geometry is minimized. In case of the calibration unit, many material, process, and environment factors influence the cooling of the material and thus can lead to undesired shrinkage and deformation. When it comes to avoiding these undesired behaviors, decision support systems are of great value to adapt the process at process time. Decision support systems require a digital interface between the profile extrusion line, servers, and databases, which many legacy machines are still lacking. Therefore, a modular, portable, and affordable measurement system is presented in the next section, before a numerical simulation model for plastic melts is introduced as a second means of gaining insights into the process.

2.1 Prerequisites for Digital Shadows

Besides classical process modeling, data acquisition is one of the key ingredients for the creation of digital shadows. On the one hand, the collected data can be used to monitor and analyze the process during operation, while on the other hand, it can be used to create data-driven models, e.g., as decision support systems. Usually, sensors are used to represent the reality in terms of specific physical values. One challenge is to select suitable data to enhance the digital shadow. Another challenge is the analysis of the measured data to gain more knowledge for further process optimization. In the following, data acquisition will be explained on the basis of profile extrusion.

New extrusion lines are already equipped with the hardware and software requirements for Industry 4.0 applications. However, due to their modular and robust structure, extrusion lines have a long life expectancy, as single parts can easily be repaired or replaced. This results in a large amount of legacy machines in production (+20 years old (Urbanek and Saal 2011)), in particular in small- and medium-sized companies, which are not equipped for data-driven problem analysis and control. Even the most basic vital signs of the extrusion process, the melt temperature, pressure, and motor load (Pilar Noriega and Rauwendaal 2019) are not routinely stored for longer-term analysis. Therefore, retrofit solutions in the form of modular measurement systems connected to a database present themselves as a convenient way to enable data collection in existing extrusion lines.

The first step was the development of such a modular, portable, and affordable measurement system, enabling the retrofit of existing analog extrusion lines, managing the general quality of the manufactured plastic parts, and collecting data for the construction of Reduced Order Models (ROMs) (Sasse and Hopmann 2021). The measurement system as shown in Fig. 3 is based on a mini computer and is capable of processing both analog and digital online and off-line data, such as the temperature (e.g., of barrel, melt, die, or extrudate), pressure (e.g., of gear pump or melt filter), rotational speed (e.g., of screw, melt pump, or haul-off), the thickness of the extrudate (by tactile, capacitive, radiometric, or optic methods), power (of cooling setup or motors of extruder or gear pump), piezoelectric sensors (for acceleration, forces, and dynamic cavity pressure), throughput (melt output or raw material input at feed hopper), or camera data. All data is processed by an Open Platform Communications (OPC) Unified Architecture (UA) client, where they are written into a common SQL-based database. Options for real-time visualization, for example, via the open-source software Grafana, are also available. Analysis of the process data is important to determine the influence of variations of process parameters and material properties (Pilar Noriega and Rauwendaal 2019), but a full problem analysis of the extrusion process is only possible with data from all timescales (milliseconds to hours/days). Saving the process data to a common database also has the side effect of added traceability of products for quality control purposes. Based on this working technical infrastructure, the measurements can be aggregated and analyzed and thus used to improve the process. In addition, the OPC UA client provides an interface for operations on external devices, such as computing clusters. This decentralized approach is a suitable method to deploy the decision support system for profile extrusion lines. Within the presented setup, a decision support system cannot only utilize archived process data from the common database but also live process data via the OPC UA client for best results.

Fig. 3

Structure of the modular, portable, and affordable measurement system

While the aforementioned strategy allows insights into the process during operation, some data cannot be acquired by measurements. This may, e.g., be the case if certain parts of the machine are not accessible for the measurement devices, or if the measurement device would influence the process operation, e.g., inserting a temperature sensor into a very small extrusion die flow channel. In these cases, numerical simulations can be used to gain additional information about the quantities of interests in the processes. The availability of stable numerical solution strategies for analyzing the physical models is thus also crucial for the creation of decision support systems. The computed numerical results can serve as high-fidelity data for training or validating the reduced models that form the core of the digital shadow of the production process. Moreover, they enable to tackle more complex tasks like optimization of machine components that cannot be solved by only considering measurement data.

One example, where the ability to accurately model the flow inside profile extruders is important, is the design of new die geometries. As the melt’s complex behavior quickly exceeds an engineer’s design intuition, the design of such dies can be accelerated by numerical optimization. The main quantity of interest here is the spatially varying velocity field, but also the pressure field and other secondary quantities like shear stresses at the walls can serve as optimization objectives (Hopmann and Michaeli 2016). Consequently, the precise numerical prediction of these quantities inside the flow channel is of great interest. To describe the viscous flow of plastic melts, the corresponding physical Partial Differential Equation (PDE) model is given by the stationary Stokes equations (conservation of momentum) and the stationary continuity equation (conservation of mass):

$$\displaystyle \begin{aligned}{} -\boldsymbol{\nabla} \cdot \mathsf{\sigma} = \mathbf{0} \qquad \text{and} \qquad \boldsymbol{\nabla} \cdot \boldsymbol{v} = \mathbf{0}, \end{aligned} $$

(1)

where v denotes the unknown velocity and σ the Cauchy stress tensor. This stress-based formulation originates from continuum mechanical considerations and is often written in terms of the unknown quantities, namely, velocity v and pressure p:

$$\displaystyle \begin{aligned}{} \mathsf{\sigma} = -p \mathsf{I} + 2 \eta \mathsf{\epsilon},\qquad \text{with}\qquad \mathsf{\epsilon} = \frac{1}{2} (\boldsymbol{\nabla} \boldsymbol{v} + \boldsymbol{\nabla} \boldsymbol{v}^T) . \end{aligned} $$

(2)

Here, η is the dynamic viscosity and 𝜖 denotes the rate-of-strain tensor. While many Computational Fluid Dynamics (CFD) applications assume a Newtonian fluid, where the viscosity is a constant material property, in our application, we need to account for the shear-thinning characteristics of molten plastic, i.e., the viscosity decreases with increasing shear rate. In order to model this behavior, there exists a variety of constitutive equations in the literature. For our application, we have chosen the Carreau-Yasuda model as it proves to be in good agreement with measurements for a wide range of shear rates (Hopmann and Michaeli 2016).

Together with the closure equations Eq. (2) and this material law, Eq. (1) forms a nonlinear PDE system, for which no closed-form analytical solution can be found. Thus, we employ the Finite Element Method (FEM) to compute a solution numerically (Tezduyar et al. 1992). Figure 4 exemplary shows the solution of a heat conduction equation computed inside an extrusion die’s flow channel on the left-hand side as well as the computational mesh, used for computing the temperature distribution on the right-hand side.

Fig. 4

Exemplary computational mesh and corresponding FEM solution of the temperature distribution in an extruded profile

Applying FEM to Eq. (1) provides high-fidelity solutions for the unknown velocity and pressure fields, which can then, e.g., be used to evaluate an objective function describing the desired target properties or constraints that have been posed on the design process. More precisely, for the design of flow channels in extrusion dies, one primary design constraint can be the homogeneity of the flow velocity at the die exit. Here, different objective functions have been proposed in the literature (Elgeti 2011; Rajkumar 2017). All of them have in common that they divide the outflow of the extrusion die into multiple patches over which an average velocity is computed. This is afterward compared to the average velocity over the whole outflow boundary. In order to compute these averages, a knowledge of the spatially varying velocity field is required, which proves the importance of proper numerical modeling for the context of shape and design optimization.

In the following section, we will present a first proof of concept, how a two-dimensional extrusion die geometry can be optimized using Reinforcement Learning.

2.2 Shape Optimization with Reinforcement Learning

Decision support systems should suggest one or multiple possible solutions to a problem. This oftentimes requires solving optimization tasks. In practical applications in real-world use cases, it is often more interesting to find reasonably good solutions in a short time than to search for the optimal solution for a long time. To accommodate this, many different approaches have been developed. Among these, RL is emerging as an additional method, whose feasibility is currently being investigated for a variety of optimization tasks (Samsonov et al. 2020, 2021; Kemmerling et al. 2020). When confronted with a set of problems which exhibit similarities, but are not identical, classical optimization approaches typically solve each problem individually, without taking advantage of the shared structure between the problems. In an RL approach, however, this shared structure can be exploited by training an agent which learns a general strategy to solve incoming problems. Once learned, this strategy can then be applied to many problems at comparatively small computational cost.

A wide variety of optimization problems emerges from production contexts, including scheduling problems, layout design of shop floors, and tool design. In this section, the focus is placed on tool design in profile extrusion settings as an illustrative example, although the approach is transferable to a wide variety of tool design tasks.

As mentioned before, one of the remaining challenges in profile extrusion is the shape optimization of flow channel geometries with the aim of minimizing shrinkage and warpage in the produced parts, hence improving product quality. Although this problem concerns the construction of tools and does not require real-time optimization capabilities as they would be needed in a production context, it demands numerical high-fidelity solutions from computationally expensive CFD simulations. This incentivizes the application of efficient optimization approaches.

In this work, the environment corresponds to the digital shadow (Bergs et al. 2021) of an extrusion die, and the agent can modify the shape of the flow channel in an iterative manner. More specifically, it can deform the computational mesh of the die geometry by moving individual control points of a spline, which is used to transform an initial mesh – a method known as Free Form Deformation (FFD) (Sederberg and Parry 1986). This type of geometry parameterization allows for a relatively low number of design parameters while simultaneously offering flexibility with respect to the resulting shapes as well as guaranteeing smooth deformations if the spline is chosen appropriately. To generate the observations, we solve the numerical model presented in Sect. 2.1 using FEM and provide the results to the agent.

The geometry modifications of the agent need to adhere to certain constraints so that the resulting tool design fulfills its intended function. To show the feasibility of the approach, the agent is trained to optimize a T-shaped geometry with one inlet and two outlets, which is inspired by the separation of the melt flow inside a coat hanger distributor. The agent’s objective is to modify this geometry such that a certain mass flow ratio between the two different outlets is achieved. During each episode, the agent is provided with a new ratio for which the geometry is supposed to be optimized. After a certain number of steps during the training, the agent is evaluated on a set of unseen goal ratios to estimate its learning and generalization progress. The results of the training and the validation of an agent trained with the Proximal Policy Optimization (PPO) algorithm (Schulman et al. 2017) are depicted in Fig. 5. As one can see, the agent learns a strategy for achieving the desired goal ratios, leading to a decrease of the number of steps per episode and an increase of the average cumulative reward per episode. Here, an incremental shape optimization strategy has been chosen, where one timestep corresponds to one action, i.e., a single modification of the geometry. After just roughly 400 episodes (corresponding to 400 different goal ratios), the agent consistently receives an average cumulative reward greater than 5 (highlighted by the area shaded in dark blue), indicating successfully accomplished goals. Even for previously unseen mass flow ratios, the agent’s performance consistently improves as shown in Fig. 5b, allowing to accomplish 4 out of 5 validation runs at the end of training.

Fig. 5

Training progress (a) and the success rate of the validation runs (b) for a PPO agent following an incremental shape optimization strategy. The area, where the cumulative reward is greater than five, is shaded in blue, indicating that the agent achieved its goal. The training is stopped after reaching 1500 episodes

The approach presented here forms the basis for further research, e.g., investigating additional, more complex geometries and incorporating more realistic optimization criteria such as the flow homogeneity.

3 Metal Processing Process Chain: Rolling, Tempering, and Fine Blanking

As already mentioned, the relationships and interactions between material, process parameters, and the final properties are not always known, although continuous processes such as rolling have been used for a long time and are well-established. Here, digital shadows in combination with various algorithms can support process experts in their decision-making process. Such decision support systems can lead to further optimization of processes and complete process chains. In this chapter, the development of digital shadows and application of data analysis and optimization algorithms are shown for selected examples of processes along a metalworking process chain. First, the steps necessary to develop a digital shadow are presented. As an example of the use of process models in the context of digital shadows, physical models are presented for hot rolling and tempering that can provide real-time information on the microstructure and thus on the final mechanical properties. Processes that follow tempering, such as press hardening, are not considered further here for the time being. Subsequently, cold rolling and fine blanking are used to demonstrate how data from processes can be integrated and processed so that conclusions can be derived about the final product properties (see Fig. 6). Finally, an exemplary decision support system is presented. Concretely, the previously presented physical model of hot rolling is combined with reinforcement learning methods to optimize process parameters.

Fig. 6

Selected processes of the metal processing process chain

3.1 Prerequisites for Digital Shadows

In this chapter, the prerequisites for developing digital shadows are presented on the basis of some selected processes of a complete metal processing process chain. More specifically, the potential of physical and semiempirical models for the prediction of product quality such as mechanical properties will be demonstrated using the example of hot rolling and tempering processes. Subsequently, it will be shown that process data analysis enriches digital shadows. Process data from cold rolling and fine blanking can be meaningfully aggregated and used to draw conclusions about material and tool behavior.

3.1.1 (Hot) Rolling + Tempering

In the following, the abovementioned processes are briefly presented, starting with rolling. Rolling is a widely used and established metal forming process employed in several industrial sectors, e.g., the automotive industry. A rolling process produces semifinished or finished products with customer specified geometry and mechanical properties. About 95 % of steel products undergo at least one rolling process during their production (Allwood et al. 2012). Rolling consists of several steps, called passes, in which the material’s thickness is reduced by moving it through two opposing rolls. In hot rolling, the material is heated above a material-specific recrystallization temperature (for steel typical 1000 ∘C–1200 ∘C) and then deformed. Therefore, the microstructure and thus product quality like the mechanical properties can be directly influenced (Lenard et al. 1999).

After rolling, the rolled materials undergo heat treatment. According to DIN 10052 (1994), heat treatment is defined as a temperature-time cycle for desired characteristics of materials or workpieces. Among all the heat treatments, quenching-tempering combination is aimed to optimize the properties, e.g., hardness and toughness of material for end use or for the following process. But the material also loses its toughness, which makes it not applicable for the further processing, therefore a reheating and holding at a temperature below a certain temperature. This process is called tempering and it is applied after quenching as a following step for regaining toughness with limited loss of hardness. Several factors have impacts on steel properties during tempering. Among them, temperature and time are the most important factors for tempering treatment. With designated temperature and proper holding time, the microstructure and thus the mechanical properties can be controlled. This affects the subsequent process such as press hardening.

First, a physical model for hot rolling is described and then for tempering. Due to the widespread use of hot rolling processes and their high complexity, modeling approaches were developed early on. The development goes back to the 1920s of the last century, where von Kármán (1925) and Siebel (1925) used basic mechanics to describe and analyze the rolling process. Based on their fundamental findings, known as the slab method, Sims published simplified equations to predict roll forces and roll torques (Sims 1954). These simplified mechanical models are able to calculate complete processes within several seconds; therefore, they are known as Fast Rolling Models (FRMs). These FRMs are often combined with semiempirical material equations to describe the microstructure. There are many similar models described in the literature. One well-known and typical FRM is SLIMMER (Beynon and Sellars 1992) which uses a thermal, a microstructure, and a mechanical model (Sims 1954) to predict roll forces, torques, and the microstructure evolution during multi-pass hot rolling. As mentioned in Sect. 1, one challenge is to predict product quality during the process. To achieve this, suitable process models must be linked with other data sources in the sense of a digital shadow. Here, an existing FRM, developed at the Institute of Metal Forming (IBF), is used for convenience (Lohmar et al. 2014). The model consists of several modules, predicting the deformation, the temperature, and the microstructure evolution as well as rolling forces and torques. However, it does not predict mechanical properties after rolling which would be essential for the prediction of the product quality. For this purpose, the model was extended by additional equations calibrated for a structural steel S355.

For the prediction of mechanical properties such as Yield Strength (YS) and Ultimate Tensile Strength (UTS), three extensions are implemented. First, the cooling of the material after rolling until the microstructure transformation of austenite γ to ferrite α is calculated. Second, at the transformation, austenite grain size d_γ is converted to a ferrite grain size d_α. Third, based on d_α and calibrated material parameters, predictions on YS and UTS are made.

For the cooling after rolling until the transformation temperature, a one-dimensional finite-difference method, which considers heat conduction inside the rolling stock, radiation, and convection on the surface, is used. For the phase transformation, from austenite d_γ to ferrite d_α, equations according to Hodgson and Gibbs (1992) and their parameters are used.

Based on d_α and Eq. (3) formulated by Hodgson and Gibbs (1992) and Singh et al. (2013), YS and UTS are predicted. YS and UTS are calculated based on the chemical composition-dependent solid solution strengthening σ_SS and grain-boundary (Hall-Petch) relationship. In addition to the YS prediction, the strain hardening due to the dislocation density $\sigma _{DIS_{YS}}$ is also accounted:

$$\displaystyle \begin{aligned}{} YS &= \sigma_{SS_{YS}} + \sigma_{DIS_{YS}} + K_{YS} \cdot d^{-0,5}_{\alpha}, \quad \text{and}\quad UTS = \sigma_{SS_{UTS}} + K_{UTS} \cdot d^{-0,5}_{\alpha} . \end{aligned} $$

(3)

Next, the FRM predictions regarding grain sizes and mechanical properties are compared to experimental data (see Fig. 7). For this, a S355 slab was heated up to 1200 ∘C and hot rolled from an initial thickness of 140 to 25 mm in eight passes. First, the average former d_γ and d_α from the experiment are compared to the FRM prediction. Both predicted grain sizes agree well with the measured ones and differ only by a few µm. Finally, tensile tests of the samples according to ISO 6895 are carried out using a Zwick Z100 testing machine. The predicted mechanical properties are 35 % higher for YS and 13 % lower for UTS. The noticeable difference in YS might be related to the simplified modeling. Furthermore, a dedicated calibration of the YS and UTS parameters for S355 structural steel should improve the accuracy of the FRM. All in all, it becomes evident that fast models can provide accurate results in real time that usefully complement digital shadows.

Fig. 7

Comparison between the measurements and FRM-predicted values

The results of this model, such as the final microstructure, can then be passed as input to the next process model so that more accurate predictions can be made. Here, hot rolling is followed by tempering. Therefore, a physical and semiempirical model for tempering is presented. Numerous tempering models are introduced in the recent years (Lee and Lee 2008; Jung et al. 2009; Smoljan et al. 2010). However, these models cover only few aspects of the quenching and tempering process with limited calculation speed, which cannot fulfill the purpose of digital shadows. Therefore, it is ideal to introduce whole scale, real-time tempering models which can be utilized for process monitoring and optimization without significant sacrifices to the accuracy.

There are several mechanisms that are contributing to the yield strength of a material. Here, the two most important are the dislocation hardening σ_disl and precipitation hardening σ_p. The dislocation density contribution is given by the Taylor equation (Kocks and Mecking 2003) accounting for the dislocation forest hardening:

$$\displaystyle \begin{aligned}{} \sigma_{\mathrm{disl}} &= \alpha M b^2 \sqrt{\rho} . \end{aligned} $$

(4)

The precipitation contribution to strength arises from the Orowan mechanism of hardening: (Smallman and Ngan 2014)

$$\displaystyle \begin{aligned}{} \sigma_{p} &= \frac{0.26\mu b}{R}\sqrt{f_p}\log\left(\frac{R}{b}\right) . \end{aligned} $$

(5)

The yield strength of the material can therefore be predicted at any point in the process by having access to the internal microstructural variables such the dislocation density ρ precipitate fraction f_p and radius R.

The change in dislocation density as a function of the lath width is given by the closed relationship (Galindo-Nava and del Castillo 2015):

$$\displaystyle \begin{aligned}{} \rho &= \frac{3E}{(1+2\nu^2)\mu}\frac{4\epsilon^2 d_{{cottrell}}}{d^2_{{lath}}b} . \end{aligned} $$

(6)

The precipitation reactions are modeled based on the precipitation phenomenology described in Deschamps and Brechet (1998). The model calculates the time evolution of the precipitate radius and the number of precipitates which can be used directly in Eq. (5). The model was evaluated for the tempering behavior of a hot rolled MMnS. The material was tempered at various temperatures and times, and the effects of the process were evaluated utilizing microhardness testing. Figure 8 shows the predicted evolution of the material hardness, for various temperatures and times, compared with the experimental measurements. The MMnS exhibits typical tempering behavior which is characterized by a steep drop-off at the beginning followed by a logarithmic reduction in the hardness. The model at its current state is able to predict quite accurately the hardness evolution at short and long times. The biggest deviations are encountered in the lower temperature regime where potentially the presence of metastable carbides is potentially not accurately predicted.

Fig. 8

Comparison of experimental and simulated hardness for tempered medium manganese steel

3.1.2 Data Analysis of the Fine Blanking Process

In addition to the model-side description over process borders, as presented earlier for hot rolling and tempering, data over several processes can also be correlated with certain properties. Here, fine blanking is chosen as an exemplary sheet metal forming process because it is the most frequently used precision cutting process in industry (Zheng et al. 2019). Fine blanking is a manufacturing process for the cutting of metallic components. It is similar to blanking in a single-stroke shearing process, but is extended by a blank holder and a counter punch to improve the quality of produced component (Klocke 2017). Areas of application for the fine blanking process arise when there are special requirements with regard to the contact area of the cut surfaces (smooth cut), the perpendicularity of the cut, as well as the achievable dimensional and form tolerances.

The process chain prior to fine blanking comprises different steps depending on the material. Hot-rolled and cold-rolled strips account for the largest share of the fine blanking material. In this example, cold-rolled material is used as an input for fine blanking (see Fig. 9). Due to the complexity of the interactions along the process chain and in the fine blanking process, the relationships between the processes are not fully understood. With an enabled IoP process, signals along the process chain can be acquired and analyzed in high quantities. The challenge is to aggregate the data and analyze it in such way that decision support can be deduced. To underline the potential in a detailed monitoring of the fine blanking process, large series of fine blanking strokes were conducted and analyzed for wear detection. The information of the tool wear at the fine blanking process will enrich the digital shadow of the fine blanking process and is a first step to enable decision support along the process chain.

Fig. 9

Process chain: cold rolling and fine blanking

For wear detection, data is acquired by nine piezoelectric sensors, which have been integrated into the tool structure to measure all process forces including punch, counterpunch, and blank holder force (Niemietz et al. 2020) and with acoustic emission sensors that have been applied to the upper pressure plate close to the punch positions (Unterberg et al. 2021). The raw sensor signal is first cleaned and subsequently segmented into the stripping phase where the sheet metal is stripped off the punch. This phase is of special importance since tensions during stripping are a primary indicator for damage of the tools, coating, and geometry. In order to model the variation in the signal of a wear-sensitive process phase, the stripping of the sheet metal off the punch, autoencoders have been utilized that are able to learn a typical force-time curve automatically (Niemietz et al. 2021). In short, a convolutional autoencoder is used to convolute and devoncolute the input time series through a bottleneck and reconstruct the original signal utilizing the mean squared error between the original input time series and the reconstructed output signal as the loss function. The presented evaluation is based on this loss function also known as reconstruction error. The hypothesis is that the development of this error over time throughout stroke series of several thousand strokes can indicate changes in the wear conditions of important tool components.

The study utilizes four experiments with about 2000–3000 strokes per experiment. For all experiments, the wear increase has been observed to be high in the first part of the experiment and near zero in the second half of the experiments.

The results presented in Fig. 10 on the left show the qualitative change of the force signals in the stripping segment over time. Stick-slip behavior can be observed for the first thousand strokes but is not observable for the later strokes. Similarly, the reconstruction error, especially for experiments 2 and 4, shows high values for the first part of the experiment but to stabilize later in the experiments. The cumulative error clearly shows the similarity to a logarithmic behavior. Both observations match the observed wear increase in the beginning and end of the experiment execution. The presented plots are found representative for several series of experiments conducted with the fine blanking process. In summary, using a very shallow noncomplex autoencoder, the amount of error in the learned encodings seem to be an indicator for wear increase during fine blanking.

Fig. 10

(a) The (cumulated) reconstruction errors are presented. (b) Selected strokes of the second experiment are presented that are representative for the change in the force signal of the stripping segments (Niemietz et al. 2021)

While the presented study is a first step to decode the dependencies within process signal variations with physical quantities of interest, the verification of the approach has to be conducted on larger and more heterogeneous experiments. Nevertheless, the force data can be used for wear detection and is thus can be used to enrich the digital shadow of the fine blanking process.

But the process is influenced by numerous other parameters such as material parameters that are defined in upstream cold rolling processes, or lubrication setup. Especially the mechanical properties of the material are varying along the coil. Typically, material properties are only measured at the beginning and end of the coil; therefore, there is no information about the exact material properties along the coil. This does not allow any conclusions to be drawn about the material quality in the event of defects in the fine blanking process. Thus, in an enabled IoP, the cold rolling-induced material variations could be monitored by monitoring the upstream processes to optimize the fine blanking process.

3.2 Process Design and Optimization with Reinforcement Learning

Due to its high industrial relevance, even small optimizations of the rolling process have a significant effect regarding energy and material consumption (Allwood et al. 2012). One factor that affects the (hot) rolling process efficiency is the process design (pass schedule) which defines all process parameters for each pass, e.g., the height-reduction and inter-pass time. Pass schedules are generated by complex heuristics designed by experts based on their experiences and with the support of FRMs or FEM simulations. Historically, pass schedules where laid out by iterative approaches where maximum allowable height reduction was applied until the desired thickness was reached (Pietrzyk et al. 1990). Additionally, there were first efforts to use genetic algorithms (Hernández Carreón et al. 2019) for multi-objective optimization.

Designing pass schedules in hot rolling processes (see Sect. 3.1.1) is a time-consuming process typically performed by domain experts. As described in Sect. 2.2, RL is a promising approach for a wide variety of optimization settings in production, including the automated design of pass schedules, where it could potentially uncover novel scheduling strategies. One of the requirements of RL is access to a simulation, since training an agent on the real process would be prohibitively costly and time-consuming. Instead of computationally expensive FEM simulations, a FRM (see Sect. 3.1.1) can be used in the hot rolling context to arrive at simulation results within seconds rather than in minutes. A given model can be enhanced with new measurements, allowing for digital shadows (Bergs et al. 2021) in the hot rolling context to be easily extended.

As in many domains, simulation models such as FRM often contain the intellectual property of the relevant stakeholders, which may be reluctant to share the details of their models because they offer a competitive advantage. When third-party machine learning experts want access to such models in the context of a World Wide Lab (WWL), this can pose a problem. However, implementing machine learning solutions does not necessarily require the FRM itself, but only access to it, i.e. the ability to query the corresponding output to input fed into the model. In the approach investigated here, this is accomplished with a Simulation-as-a-Service (SaaS) architecture first introduced in Scheiderer et al. (2020), where a suitable interface is deployed on the infrastructure of rolling mill operators, which can access the FRM internally and can provide simulation outputs when queried by other stakeholders within the WWL. Beyond the scope of this use case, such SaaS architectures can generally serve as enablers to the IoP wherever giving third parties access to the internals of a simulation model is undesirable, but providing access to simulation inputs and outputs is not problematic.

The proposed architecture is validated (Scheiderer et al. 2020) by training a SAC agent (Haarnoja et al. 2018) to create pass schedules of a fixed length by controlling the roll gap and pause time in a hot rolling scenario. Rather than providing the agent with direct access to the simulation, it indirectly interacts with it through the SaaS architecture. The agent trained in this way shows good convergence behavior as shown in Fig. 11a and generates reasonably good pass schedules compared to those created by domain experts (see Fig. 11b and c). While this work shows the general feasibility of the approach, many opportunities for improvement remain unexploited. These include improving upon the current design of the reward function, learning from pass schedules generated by experts and investigating the transfer of trained agents to scenarios with different material properties.

Fig. 11

(a) Training progress of a Soft Actor-Critic (SAC) agent on the rolling task shown as the rolling mean of the received reward with the shaded region showing the rolling standard deviation. (b) Height and (c) grain size of material throughout pass schedule generated by a trained RL agent and a pass schedule created by a process expert. (Adapted from Scheiderer et al. 2020)

Next to section Sect. 2.2, where the applicability of RL for tool design was demonstrated, this section additionally shows the potential of RL in process design as exemplified by hot rolling scheduling. The ability to produce schedules similar to those created by human operators very quickly can be valuable in many other process design problems as well. Ultimately, such approaches may serve to realize the vision of the IoP by enabling networks of virtual agents, which can dynamically respond to changing requirements from other components in the network.

4 Conclusion and Outlook

In this contribution, throughout two different process chains with continuous processes, different stages are presented that enable decision support. A suitable basis for this are digital shadows. Therefore, the development of the digital shadows is first given by a detailed description of numerical, physical, and semiempirical models. For instance, rolling and tempering models are shown that predict in real-time product properties such as yield strength or hardness. The comparison with measurement shows that the results of these models can make a valuable contribution to digital shadows. However, the parameters for the semiempirical equations have to be determined very precisely; otherwise, the predictions deviate too much.

In addition to models, digital shadows need process data in real time. Therefore, a modular concept for data acquisition, aggregation, and processing is demonstrated on the example of a typical extrusion line. The concept shows how both analog and digital signals can be recorded with simple and low-cost equipment and systematically stored in a database using the well-known standard OPC UA. Systematically stored process data is very useful, as shown by the example of fine blanking. Based on the time-series data from the stamping force sensors, convolutional autoencoder is used to extract patterns in order to detect wear. The results show that such a data-driven approach is promising and generally suitable to study the highly complex interactions between process parameters and product quality. For this reason, further research is ongoing with the aim of using the insights gained for process optimization.

For a decision support system, however, more is needed. For both process chains, model data is coupled with reinforcement learning to optimize the shape of either process components or process parameters. For the profile extrusion use case, the optimization of a T-shaped geometry represents a flow channel in profile extrusion in order to minimize shrinkage and warpage. The algorithm modifies the geometry of the flow channel by moving individual points of a spline. In the future, the approach will be extended to include more complex geometries and incorporate flow homogeneity as optimization criteria.

In summary, it can be stated that the general approach is independent of the process. Especially the optimizations by coupling model data with reinforcement learning show this very clearly. The concrete implementation, however, requires detailed process knowledge and is only directly transferable to other processes to a limited extent.