Digital twin-driven smelting process management method for converter steelmaking

Abstract

The converter is an indispensable key equipment in the steel manufacturing industry. With the increasing demand for high-quality steel, there is an increasing demand for monitoring and controlling the status of the converter during the smelting process. Compared to other manufacturing industries, such as food processing and textile, converter steelmaking requires a larger keep-out zone due to its ultra-high temperatures and harsh smelting environment. This makes it difficult for personnel to fully understand, analyze, and manage the smelting process, resulting in low production efficiency and the inability to achieve consistently high-quality results. Aiming at the low virtual visualization level and insufficient monitoring ability of the converter steelmaking process, a process management method based on digital twin technology is proposed. Firstly, a digital twin system framework for full-process monitoring of converter steelmaking is proposed based on the analysis of the process characteristics of converter steelmaking. The proposed framework provides critical enabling technologies such as point cloud-based digital twin model construction, visual display, and steel endpoint analysis and prediction, to support full-process, high-fidelity intelligent monitoring. After conducting experiments, a digital twin-driven smelting process management system was developed to manage the entire smelting process. The system has proven to be effective as it increased the monthly production capacity by 77.7%. The waste of smelting materials has also been greatly reduced from 34% without the system to 7.8% with the system. Based on these results, it is evident that this system significantly enhances smelting efficiency and reduces both the costs and waste associated with the process.

Introduction

The world is surpassing mass customization, and the manufacturing industry has already achieved mass customization ahead of other industries (Aheleroff et al., 2022). As a fundamental material, steel’s high strength and compressive resistance accord it an essential position in many engineering constructions, broadly applied in bridges, roads, buildings, and machine manufacture. In recent years, rapid economic expansion has increased steel demand, consequently focusing attention on the steelmaking industry. The converter, an essential equipment in the process of steel smelting, not only adjusts the temperature of steel, improving its organization and physical performance, but also facilitates the purification and preparation of steel. Additionally, the converter has the advantages of a significant temperature increment and shortened duration, greatly improving steel smelting efficiency and quality of steel. Accordingly, it holds great importance in the effort to enhance steel smelting level, optimize steel quality, increase process utilization, and reduce energy consumption. Despite this, the steel smelting environment is harsh, the cost is high, and equipment failures can lead to waste of raw materials and poor final product quality. Thus, measuring the thickness of the converter lining and real-time detection of the smelting state, especially regarding the condition of the converter, are important steps to avoid raw material wastage and poor quality of final products.

As the automatic detection technology progresses, many new methods for steel refining monitoring have emerged, such as using optical information at the taphole flame for endpoint control and laser scanning of the taphole lining for measuring thickness (Birk et al., 2002; Han et al., 2020; Zhao et al., 2013). However, current steel refining automatic control technologies still cannot remotely or intuitively observe the converter’s smelting process and usage status, and its equipment and maintenance costs remain relatively high. At the same time, complex and redundant digital expression training brings high learning costs and time costs to workers, which can easily lead to tremendous errors in smelting results and cause serious consequences. Therefore, ensuring the long-term reliable operation of equipment requires not only full consideration of factors that affect equipment operation, but also precise process management methods.

The definition of Process Management is a management method used to plan and implement various business processes in an organization. DT has been proven to bring advantages to industries as a critical Virtual-Real Fusion technology (Aheleroff et al., 2021). Focusing on the practical needs of steelmaking, the DT technology holds the following advantages:

• (1) The extremely hostile smelting environment makes direct contact with converters difficult for personnel, thus causing a lack of real-time monitoring of the smelting process. DT technology can allow direct control of the production process by mapping various physical objects and digital models. This could help reduce personnel’s time in this hazardous environment.

• (2) DT technology often uses IoT technology for data collection, cloud technology for data storage, and big data technology for data analysis. It provides strong support for the optimization and adjustment of industrial production.

• (3) The steelmaking process involves complex thermodynamic and chemical reactions. Accurate monitoring of converter status and prediction of steelmaking endpoint is crucial for obtaining steel quality (Wang et al., 2020). DT technology allows real-time monitoring of the molten steel condition within the converter, which realizes control of the molten steel out of the converter at the end of the steelmaking process. It can minimize various issues arising during steelmaking and reduce unnecessary top-blowing operations.

In conclusion, DT technology has great potential application in converter steelmaking. Process management involves analyzing process data to provide workers with information for controlling the manufacturing process. Data monitoring involves collecting and presenting data to users in an understandable manner. DT technology applies to the control and monitoring of the smelting process of converter steelmaking. It also realizes real-time perception, reliable transmission, and intelligent processing of physical world information. Integrating DT technology with field sensor data enables intelligent and automated manufacturing for converter steelmaking control. Realizing the intelligence of converter steelmaking has the following important significance:

• (1) Enable real-time monitoring of the steelmaking process, allowing remote workers to interact safely. It can prevent issues such as splattering, rebound drying, and others caused by insufficient experience and uncertainty in manual operations, thus ensuring the quality of steel production.

• (2) An efficient and cost-effective method has been developed to improve the endpoint hitting rate in the ladle refining process. This method reduces the number of tuyere blowing, which shortens the refining time, increases production efficiency, and lowers costs.

• (3) Timely monitoring of converter usage and smelting status improves steelmaking intelligence and reduces labor intensity.

Based on analysis, we propose a DT-based converter steelmaking process management system to solve intelligent and digital visualization issues in harsh environments. By constructing the high-fidelity DT body of the large converter equipment, multiple dynamic data in the smelting process can be collected and fused. Finally, real-time modeling, multiple source historical data storage, and real-time dynamic human–computer interaction are realized.

”Related works” Sect. introduces the research related to DT applications and converter lining thickness measurements in the industry and summarizes the research gaps. ”Framework for the DT system” Sect. mainly introduces the DT system framework for the converter steelmaking process. ”Geometry information perception and calculation” Sect. introduces the construction of the DT model and the method of monitoring converter lining thickness. ”Physical information perception and prediction” Sect. mainly introduces the methods of data collection, communication, and prediction of the steelmaking endpoint. ”System implementation and verification” Sect. verifies the proposed approach through some experiments. Lastly, ”Conclusion” Sect. is the conclusion of the entire paper.

Related works

This Section will delve into an analysis of DT technology and process management technology for the converter smelting process in order to bring to light the existing research gaps.

DT in industry

In recent years, with the rapid development of intelligent manufacturing the global manufacturing technology system and manufacturing mode have undergone tremendous changes. DT technology, one of the emerging technologies of intelligent manufacturing, is the entry point for integrating industrialization and informatization (Lv et al., 2023). It has generated many intelligent devices, software, and systems. DT technology effectively uses models and data to achieve effective interconnection between the physical and digital virtual worlds (AboElHassan & Yacout, 2023; Liu et al., 2023a, 2023b). Lim et al. (2020) summarized the application prospect of DT by investigating the current situation and providing guidance for developing and applying industrial technology. Singh et al. (2023) proposed that industrial machine monitoring and diagnosis are very important in industrial 4.0 and reviewed the application of artificial intelligence in fault diagnosis. Redelinghuys et al., (2020) proposed a DT architecture for manufacturing systems in the context of Industry 4.0, CPPS, and the IoT. With the gradual development of key technologies involved in the theoretical system of DT, DT has been applied to many intelligent industrial manufacturing processes (Jones et al., 2020; Josifovska et al., 2019; Qamsane et al., 2021; Wilhelm et al., 2021). The three key technologies involved in virtual scene visualization and process control in the manufacturing industry are as follows.

Regarding DT modeling and simulation, Kimmel et al. (2016) achieved good accuracy and quality in 3D modeling based on point clouds by using multi-source image datasets and fast spherical feature extraction algorithms. Neumann et al. (2018) proposed a deep convolutional neural network architecture for real-time object modeling, which can reconstruct 3D models with pixel-level accuracy and fast operation. Bai et al. (2018) proposed an object modeling technique based on generative adversarial networks, which can quickly generate fine 3D models with high accuracy. All of the aforementioned scholars adopted various approaches to model construction; however, due to the inclusion of a great amount of data in the steelmaking process, they all lack optimization and processing of the large-scale model data, leading to slow system operation, which hampers monitoring performance.

Regarding dynamic data sensing and storage, Zhou et al. (2013) proposed a novel data acquisition system based on microcontrollers, which collects physical parameters such as temperature, humidity, ATM pressure, etc., and stores the collected data in the microcontroller’s internal memory. Wang et al. (2014) developed a data acquisition and storage system based on ARM processors. This system uses ARM processors as the main control core, coupled with software-defined networking technology, which can collect and store a large number of complex dynamic data in real-time, and has good reliability and real-time performance. In addition, some researchers have focused on the application of SDN technology in dynamic data acquisition and storage. For example, Li et al. (2017) proposed a system architecture of dynamic data acquisition and storage based on SDN technology. This architecture can intelligently collect and store a large amount of dynamic data while maintaining good real-time performance, reliability, and scalability. These methods can achieve dynamic data acquisition and storage, but in the process of converter steelmaking, they cannot fully collect the data types required, and there are some defects in terms of data stability and diversity, and it is easy to cause data loss. Additionally, these methods are costly to install, debug, and maintain, making them unsuitable for steel mills.

Regarding human–computer interaction and visualization, Pimenov et al. (2022) proposed an artificial intelligence-based tool status visualization monitoring system and discussed the future prospects of AI visualization. Lewsey et al. (2022) proposed a cloud-based visualization modeling technique that can better support users in analyzing and visualizing complex datasets. Additionally, Mal et al. (2023) utilized virtual reality technology to implement a spatial relationship-based graphic visualization method that effectively improves the usability and interactivity of visualization technology. Wan et al. (2019) proposed an intelligent human–machine interaction technology based on a recommendation engine, which can construct an intelligent “knowledge map” in the software interface to support user operations better.

Currently, there is a lack of research on visualization and process management of steelmaking in harsh environments. This has resulted in several issues such as inadequate visualization, poor interactivity, and inaccurate predictions. Therefore, it is necessary to optimize and enhance the user experience to address these problems.

Thickness measurement of converter lining

Cemernek et al. (2022) proposed data monitoring directions in steel continuous casting production, providing data monitoring directions for ladle refining processes. Currently, there are many methods for measuring the ladle lining thickness, which include radiometric isotopes, impact wave measurement, thermal measurement, and core drilling. Relevant scholars have developed several ladle lining thickness measurement-related methods, and achieved certain results (Ge et al., 2020). Temperature measurement is one of the most popular methods for measuring the profile of the inner surface of the ladle, which is based on numerical solutions for the inverse problem of using thermal models and thermocouple readings (Zhang et al., 2013). However, temperature measurement requires iterative solutions to obtain the domain boundaries, leading to huge calculations and insufficient accuracy. The radiometric method uses the gamma ray emitted from the radiation source to the ladle inner. The function of scattering intensity can obtain the thickness of the liner. In addition, the core drill method is invasive, which requires obtaining a part of the sample inside the ladle to check the thickness of a part of the wall. Sadri (Sadri, 2008; Sadri & Gebski, 2011) proposed the AU-E technique, which uses the impact echo technique to detect ladle lining thickness data. The literature also proposes the influence of temperature, ladle shape, and size on wave velocity.

However, the above-mentioned method only provides localized smelting information or rough estimates of the refractory thickness, which cannot guarantee adequate radiation safety, accuracy, and precision.

Research gaps

The DT system includes simulation, vision, interaction, and modeling: using collected data to simulate and predict on the model, and providing visual feedback to users for interaction. From the above analysis, the following issues can be observed in the smelting process of the current steel manufacturing industry:

• (1) Visualization: At present, traditional measurements fail to accurately gauge the thickness of the hearth lining, failing to reach the goal of precise maintenance of the converter equipment.

• (2) Interaction: As of yet, no DT technology has been applied in the metallurgical processes under harsh environments, which means that personnel cannot intuitively observe smelting and control the process in harsh metallurgical environments

• (3) Monitoring and predicting: Due to existing detecting systems only focusing on one module’s data, it fails to grasp all related data during the metallurgical process accurately. It is impossible to analyze and predict global data for efficiency improvement.

These aspects are crucial for monitoring and predicting steelmaking process control in harsh environments, which this paper will further discuss in the following Section.

Framework for the DT system

This Section proposes a DT framework for the steel smelting process to provide an environment for virtual-real interaction and a basis for experiments.

Analysis of converters manufacturing process

As illustrated in Fig. 1, the smelting cycle of converter steelmaking is divided into four main processes. The process encompasses several phases, including loading, blowing, endpoint control, deoxidation of steel, slag splashing, slag tapping, and so on.

• (1) At the onset of the smelting process, the quantity of molten iron, pig iron, scrap steel and slag materials required for the calculation is determined according to a material-energy balance model, whereupon the materials are added.

• (2) The main process of smelting is the blowing oxygen process. The entire blowing process is controlled by adjusting the gun position height and gun pressure of the oxygen gun. Oxygen is injected vertically downwards from the iron liquid surface with high pressure from the oxygen gun. The slag material is covered on the surface of the iron liquid, which contacts the iron liquid and oxygen. Utilizing the oxidation of slag material to realize the reactions of de-carbonization, dephosphorization and desulfurization to remove impurities in steel liquid. According to the requirements of the converter and the type of steel to be smelted, the endpoint determination is made according to the control methods and manual judgment adopted.

• (3) Once the blowing of oxygen was terminated, the temperature of the steel sample retrieved through the rotating converter mouth was measured. If the temperature of the molten steel was found to be below the requirement, oxygen would then be replenished. Sampling and analysis of the steel would be executed when the temperature exceeded the lowest casting requirement. Finally, as long as the analysis result satisfied the type requirements of the steel being smelted, casting would then be performed.

• (4) Perform the steel discharge operation. Pouring the steel out of the converter.

Fig. 1

Converter steelmaking operation process

The temperature and oxygen gun data are predominantly determinants of controlling the process of blowing oxygen and predicting the endpoint of the converter, consequently deciding the steel quality. The endpoint of steelmaking is the key prediction object of the management system in this article.

In a practical smelting process, comprehensive dimensional data of the smelting process must be controlled. Grasping the complete smelting data can not only improve the quality of actual smelting, but also ensure the safety of personnel, and reduce the cost of equipment use.

Framework of DT manufacturing system for converter steelmaking

A Digital Twin is a virtual replica of a physical object that creates a two-way mapping between the physical entity and its digital model (Aheleroff et al., 2021; Fu et al., 2024; Zheng et al., 2024; Zhang et al., 2022). Referring to the closed-loop structure of ISO 23247, this system achieves process management of smelting status through digital dual drives, and implements effective interaction mode based on the HCPS framework. As illustrated in Fig. 2, the framework of the converter steelmaking process equipment and operational status management system based on DT technology includes two layers: physical space and virtual space. Two spaces and personnel collaborate with each other to form the physical-cyber-human system (HCPS).

Fig. 2

Framework of DT-based HCPS management system for converter steelmaking process

The physical layer of the system refers to the entity equipment existing in the converter steelmaking process, including the main body of the converter, sensors and tools. The tools contain positioning tools, scanning devices, and detection devices. Scanning devices are used to scan the 3D point cloud data of the converter for easy 3D modeling. 3D Point cloud data refers to a collection of points in a three-dimensional coordinate system. Each point contains three-dimensional coordinates, as well as color information(RGB) or reflection intensity information(Intensity). Positioning tools are mainly used to locate and align the data scanned. Sensors and detection devices mainly collect the dynamic data of the converter during the smelting process.

As a physical-layer mirroring system that facilitates users’ monitoring and observation, a virtual space must be in real-time sync with the physical space and reflect its related characteristics. The virtual space consists of four layers, namely, the data layer, the model layer, the application layer, and the service layer.

• (1) The data layer is mainly used to manage the state data and running data of the physical model obtained by the collection. Its historical records format enables users to monitor the data at any time.

• (2) At the model level, construct a geometric model of the converter based on 3D point cloud data. In particular, this DT model is a highly realistic digital description model that simulates the geometric shape of the converter while including detailed internal and external shapes of the converter.

• (3) The system gathers data and consolidates it. This data enables workers to monitor the converter’s operating status, determine whether the converter lining requires fixing, and even predict the end-time of the steelmaking process.

• (4) The application layer is the geometric implementation between the above two layers for virtual visualization monitoring and running status monitoring. Use UNITY to encapsulate the data, algorithms, simulations, and results of the DT process, transforming the diagnosis of equipment operation status and smelting status into an easy-to-understand visual presentation. It describes the virtual scenes and the geometric and physical states of operational data, including the tilt angle of the converter, the thickness of the converter lining, changes in the pouring state of molten steel, and related predictive data.

• (5) The service layer is the upstream of the application layer and the embodiment of the application functions. It directly participates in monitoring and managing the ladle steelmaking process by PC, Phone, Pad, Web and VR clients.

The key technologies to establish a large-scale converter steelmaking process management system based on DT technology are the construction of DT virtual models, extraction of multi-source dynamic data, and virtual interactive mapping. This paper will first build highly realistic geometry, behavior, and rule models in the following research. Then, information acquisition, transmission, and storage will extract multi-source dynamic data. Finally, the DT system will be set up and operated to achieve the goal of dynamic management process variables.

Process management for converter steelmaking

A DT-based process management system is proposed to visualize the operation state of steelmaking process equipment under harsh environments, addressing the lack of digitization and intelligence in this field. This system requires two foundation components and a human-based visualized module. It is constructed by Geometry Perception and Physical Perception, as shown in Fig. 3. A high-fidelity DT of the large-scale steelmaking equipment is built to achieve geometry perception. In addition, multi-dimensional dynamic data in the smelting process is collected and integrated to achieve physical data perception. Finally, features such as real-time mapping, storing of multi-source historical data, and real-time dynamic human–computer interaction are realized. The details are as follows.

• (1) Geometric information perception and calculation: DT geometric models of large-scale converters are built by collecting, aligning and surface processing of point cloud data. The system immediately prompts for converter lining repair based on monitored thickness.

• (2) Physical information perception and prediction: utilizes various functional sensing devices to acquire related physical data, and transmits these data via communication protocols. The collected data is processed in the DT system, and the rule action is applied to the geometric model, allowing users to observe the physical dynamic data changes intuitively. The system integrates data to predict steelmaking endpoint time and monitor converter status.

• (3) The HVM was developed to construct a comprehensive visualization system for the steelmaking process, comprising static scene modules, view switch modules and converter detail data view modules, to monitor and control the data details during the smelting process.

Fig. 3

The process of building an intelligent inspection system for converter steelmaking

Geometry information perception and calculation

Virtual models are an essential component of DT systems, thus, corresponding twin models must be built according to physical entities. Aiming at multi-angle point cloud model registration, this Section proposes an improved ICP algorithm to address the unreasonable situation of point cloud data registration under different angles. Furthermore, behavior and rule models and human–computer interaction functions are constructed to construct a static scene. As shown in Fig. 4, the DT model construction process is illustrated.

Fig. 4

Model construction Process converter point cloud data processing

Three-dimensional laser scanners were used to measure the exterior surfaces of the converter from multiple angles, thereby enabling the overall modeling of the converter. Subsequently, noises and abnormal points in the point cloud had to be removed (Qu et al., 2023). The high precision of the scanners resulted in an excessive amount of point cloud data. Therefore, the large-scale point cloud data had to be filtered and simplified while maintaining its features.

Noise, outliers and anomalies are considered useless information. After acquiring the data, statistical algorithms are first used to remove noise, outliers and anomalies (Compagnoni et al., 2017). The average distance from each point to its nearest k neighbor points is then calculated. The mean and variance of the distances are obtained, followed by the elimination of issues beyond 3σ.

The voxelization of a 3D space into many smaller voxels can be utilized to achieve down-sampling of the original data. The data set created by voxel-based filtering is not a subset of the original data; instead, it can maintain the geometric structure of the original data while filtering it properly.

Improved ICP alignment algorithm

Each frame scanned by the scanner provides a 2.5D view of a certain range in front of the scanner. Complete 3D models can be generated by stitching together multiple 2.5D views taken from different angles. Generally speaking, PCR refers to the estimation of the transformation between two overlapping point cloud datasets obtained by the scanning distance, followed by best matching and alignment. Usually, PCR combines geometric and color information to improve the registration effect. This paper mainly carries out registration based on geometric conditions. The improved ICP algorithm is used to compute the correspondent relationship between the point clouds dataset acquired from different angles.

Traditional ICP algorithm (Cong et al., 2022) requires that the two datasets have approximately equal numbers of points, and that each point in one set has a corresponding point in the other. However, this could lead to failure when there is noise and outliers, or if there is a lack of corresponding points between the two datasets. The flow chart in Fig. 5 represents the improved ICP algorithm that utilizes a voting mechanism to select the optimal alignment scheme.

Fig. 5

Improved ICP algorithm process

In order to improve the direct sample-based feature matching, the uneven surface and inner lining of the converter cause unsatisfactory results. Consequently, the surface features are computed using a MLS polynomial. Let ${p}_{x}$ denote the local neighborhood of the surface of a given point x for every MLS polynomial. To ensure that the reversed (n, u, v) coordinate does not change, the local coordinates (n, u, v) of the data points are modified by replacing their coefficients with absolute values. If the modified coefficients match, the two data points are considered candidate points. Figure 6 illustrates a data-driven approach for quantifying the difference between data vertices and model vertices given point sets ${R}_{A}$ and ${R}_{B}$; with each vertex ${R}_{A}^{i}\in {R}_{A}$, the MLS features calculate up to m potential candidate matches from ${R}_{B}$.

Fig. 6

Candidate point bifurcation diagram

The Improved ICP algorithm utilizes a voting accumulator (Zhang et al., 2021) and an incremental approach to allocate strength values for candidate matches and perform an alignment. Denote the set of data vertices having at least one candidate match as A, with B denoting the set of candidates matches of vertices in A:

• (1) The selection of proximate points which can communicate with each other is proposed. For dataset A, a data set with radius R was built around each point, and the MLS features were employed to calculate at most m potential matching points.

• (2) For the given vertex x and its neighbor f, there are m candidate pairs of matched points for x and m candidates for f. A specific set of candidate pairs {(x, xi), (f, fj)} can be given, whose relative distance will not change under rigid transformation. The Euclidean distance between two neighboring points is ${d}_{xf}$, and the corresponding Euclidean distance between two candidate points is ${d}_{ij}$.

• (3) The inconsistency between two candidate matches is represented by $|{d}_{xf}-{d}_{ij}|$, and the normalized inconsistency is measured by NMI: $NMI=(|d\_xf-d\_ij |)/(|d\_xf+d\_ij |)$. Correspondingly, NMC is defined as 1-NMI. When two candidates are perfectly matched, NMC equals 1. We assume the NMC threshold is 0.98 to indicate sufficient candidate matching consistency. If NMC is greater than the predefined threshold, x and f will respectively vote for each other candidate matching. The voting scope is equal to the corresponding NMC. The overall strength of a candidate ${x}_{i}$ voted by its neighboring point f is denoted by strength(${x}_{i}$,f), and the total strength of ${x}_{i}$ is strength (${x}_{i}$).

• (4) Points that do not meet the criteria of $strength\left({x}_{i}\right)<\Gamma *{{\text{F}}}_{c}*{{\text{F}}}_{x}$ are excluded. Here Γ is the minimum consistent threshold, F is the predefined number of neighboring points, and ${{\text{F}}}_{x}$ is the actual number of neighboring points. ${{\text{F}}}_{c}$ is the minimum number of neighboring points within the threshold of [0,1], and is not counted in strength for neighbor points not assigned by vertices during the allocation phase.

• (5) Calculating the maximum weight $w\left(M\right)=\sum_{E\in M}w(E)$ for solving a bipartite graph G (V, E) for translating and rotating between two datasets, where V is the dataset vertices and E are the edges connecting between the datasets, and M is a subset of edges that does not have two edges incident on the same vertex.

Point cloud model processing module

After the point cloud data were registered, they could not be directly utilized to simulate the surface of the converter, necessitating subsequent processing to replicate the surface state of the converter, so as to facilitate direct visualization and, hence, observation by personnel. A Least Squares Improved B-Spline point cloud-based surface reconstruction algorithm was adopted to express the surface. The algorithm could furnish a more precise reconstruction of the model.

As illustrated in Fig. 7, a model surface processing algorithm based on B-spline was employed for surface fitting. The algorithm can be divided into two parts: first, an interpolation function was determined to construct a minimal rectangular domain covering all point clouds, and a grid was subsequently divided. Then, a linear interpolation method was employed to approximate the original point clouds.

Fig. 7

Model surface treatment algorithm

A two-dimensional system of k-th order B-splines can be represented as ${Z}_{k}=f({X}_{k},{Y}_{k})$, where N points of cloud data points ${f}_{k}=f(X,Y)$ are known. The cloud data can be represented as $p=\{({X}_{k},{Y}_{k},{Z}_{k})|{Z}_{k}=f({X}_{k},{Y}_{k})\}$. The planar control point network matrix is constructed and the interpolation points (${X}_{k},{Y}_{k}$) are projected onto the XoY plane. The cloud data points ${p}_{ij}({X}_{k},{Y}_{k},{Z}_{k})$ in the corresponding subdomain are represented by B-spline interpolation surface functions. According to the interpolation points ${({X}_{k},{Y}_{k})}_{i,j}$, the corresponding cloud data points ${p}_{ij}\left({X}_{k},{Y}_{k},{Z}_{k}\right)$ are calculated. In the grid division process, the extremal values of point P are marked by ${X}_{max}$、${X}_{min}$、${Y}_{max}$ and ${Y}_{min}$, respectively. The grid typically includes N cloud data points, and the grid units in the horizontal and vertical coordinate directions are divided by formula 1.

$$\left\{\begin{array}{c}{X}_{d}=int(\frac{{X}_{max}-{X}_{min}}{{d}_{0}})+1\\ {Y}_{d}=int(\frac{{Y}_{max}-{Y}_{min}}{{d}_{0}})+1\end{array}\right.$$

(1)

Let ${d}_{0}=\sqrt{\frac{{(X}_{max}-{X}_{min})({Y}_{max}-{Y}_{min})}{N}}$. Assuming all control vertices are contained within a rectangular domain, the domain is then sub-divided into a matrix of grid points, where each grid point is denoted as ${C}_{ij}$, wherein i represents the row number and j the column number. As all point cloud data are contained within the matrix region, the grid values of the control vertices within the rectangular domain are computed in order to implement the reconstruction of the B-spline surface.

Calculation of converter lining thickness

To obtain the required measurement point in the converter coordinate system and the thickness information of the converter lining at that location, all scanner data coordinates need to be transformed into a coordinate system with the converter coordinate origin as the origin. This will then be compared and calculated with the inner wall model of the converter steel shell to obtain the necessary coordinate and thickness information.

After performing a coordinate transformation, we obtain the coordinates of the scanner in the far point coordinate system within the converter (X, Y, Z). We then derive the coordinate system relative to the scanner (${X}_{1},{Y}_{1},{Z}_{1}$) using the same transformation. Using the transformed coordinates, we can determine the location of the test point relative to the converter origin $\left({X}_{0},{Y}_{0},{Z}_{0}\right)=(X+{X}_{1},Y+{Y}_{1},Z+{Z}_{1})$ by adding the scanner’s relative coordinates to the far point coordinates. We can then calculate the position of the target point in the original converter model by combining the rotation angle of the converter, θ. Formula 2 is then used to determine the thickness of the converter lining at the target point. The system uses different colors to represent the various thickness information.

$$\left(\genfrac{}{}{0pt}{}{\left(X+{X}_{1}\right)}{\begin{array}{c}\sqrt{{\left(Y+{Y}_{1}\right)}^{2}+{\left(Z+{Z}_{1}\right)}^{2}}*{\text{cos}}\left({sin}^{-1}\left(\frac{Y+{Y}_{1}}{\sqrt{{\left(Y+{Y}_{1}\right)}^{2}+{\left(Z+{Z}_{1}\right)}^{2}}}\right)+\uptheta \right)\\ \sqrt{{\left(Y+{Y}_{1}\right)}^{2}+{\left(Z+{Z}_{1}\right)}^{2}}*{\text{sin}}\left({sin}^{-1}\left(\frac{Y+{Y}_{1}}{\sqrt{{\left(Y+{Y}_{1}\right)}^{2}+{\left(Z+{Z}_{1}\right)}^{2}}}\right)+\uptheta \right)\end{array}}\right)$$

(2)

Upon scanning the converter, the system prompts for repairs and staff fills the damaged lining accordingly. This step was taken to increase the converter’s service life and decrease the maintenance and smelting costs. The age of the converter and the lifespan of its lining have a significant impact on the factory’s economic benefits. It not only affects the ton steel consumption but also influences the converter’s productivity.

Physical information perception and prediction

This Section employs various functional sensors to acquire associated data and process it via relevant protocols in order to achieve full-process monitoring and prediction of the steelmaking converter states, as illustrated by Fig. 8, which is the physical information perception process.

Fig. 8

Physical information perception process

Multi-source data acquisition

This paper aims to collect temperature, vibration, and other necessary monitoring data of the smelting process of the converter through different sensors.

Temperature acquisition

The principle of DCTM is based on the variation of energy ratio between two wavelengths emitted by the tested object when the temperature changes. DCTM determines the temperature of the tested object by receiving the ratio of the energy with two different wavelengths. Generally, two monochromatic filters with narrow bandwidths are used to determine two wavelengths. After the energy of the two different wavelengths is transformed into electrical signals, a numerical comparison is conducted, and the ratio finally determines the temperature. The advantages of DCTM are as follows.

• (1) DCTM can mitigate the influences of water vapor, dust, variation in the size of the detection target, partial occlusion and emissivity.

• (2) Facilitating the measurement of a majority of graybody materials without correcting the dual color coefficient, and requiring only an average of the highest temperature within a given area.

• (3) Even when the detected signal attenuated by 95%, there was no impact on the temperature measurement results.

• (4) Oxidation layers can be overcome to influence temperature detection.

The double-color temperature measurement principle applies to the harsh conditions of the steelmaking process, and can maintain a high degree of accuracy. When the converter launder is rotated to a fixed position, a double-color infrared thermometer is used to target the position of the steel bath and collect temperature data of the steel bath in real-time.

Vibration data acquisition

During the blowing process of the converter, high-pressure oxygen flow is sprayed out from the oxygen gun to blow the molten pool. In this process, the oxygen gun is affected by the reactive force of the airflow, the buoyancy of the slag and the impact force of the foam slag stirring, resulting in vibration. Depending on the slag state, the vibrational force acting on the oxygen gun is also different. Usually, the result of these forces acting together is that the oxygen gun produces low-frequency vibration, the amplitude of which is related to the height of the slag position. The higher the slag position is, the greater the buoyancy and impact force of the oxygen gun. Therefore, the vibration of the oxygen gun is mainly related to oxygen pressure, oxygen gun position and slag position height. By using the known oxygen pressure and oxygen gun position smelting information to collect relevant data, the distance between the oxygen gun and the slag position, and the corresponding relationship between the oxygen pressure and the amplitude can be established. Then, using the established relationship, the measured amplitude and the known oxygen pressure, the distance can be calculated to calculate the slag level height to master the smelting process information of the converter.

The arrangement of vibration sensors is of particular importance to monitoring the vibration of a lance. In this system, the lance is mounted on a lifting trolley and a winch controls the lifting trolley. The lifting trolley can travel along a fixed guide rail. Under the guidance of the fixed guide rail device, not only can the lance move vertically, but also reduces the body vibration of the lance caused by the airflow impact during the blowing process.

The lifting trolley consists of a car frame, wheels, brakes and other components. In order to achieve better vibration monitoring, sensors should be placed in positions with moderate vibration. In this system, three sensors are respectively placed at the following two positions:

• (1) The first sensor was fixed on the lifting cart platform.

• (2) The second frame of the lifting trolley was located in the middle of the oxygen gun, which was relatively appropriate. Thus, two vibration sensors were arranged and horizontally fixed on the second frame of the lifting trolley.

Due to the fixed attachment of the oxygen gun to the lifting trolley frame, the vibration of the frame is directly related to the vibration of the oxygen gun, meeting the requirements of the monitoring system.

Other data acquisition

This study installed an inclinometer and rotary sensors at the converter’s relative locations to collect the converter’s inclination and rotating data.

Data processing

Digital signal transmission is employed in the system to facilitate the direct processing of digital signals to obtain the final result.

Infrared temperature and spectroscopic sensors are connected directly to the DT system through serial and optical fibers. After acquiring the steel temperature data, analog-to-digital conversion is conducted, followed by communication protocol data transmission. The DT system processes the data to obtain the steel temperature in a steady state.

The integration of a signal acquisition-amplifying circuit within the hardware system of a temperature sensor was explored. The process of temperature information acquisition involved the following steps:

• (1) The thermometer will convert the infrared radiation received into an electrical signal.

• (2) Upon comparison and amplification, two analog signals, 4-20 mA electrical and digital signals, are produced. The analog signal is further processed into a digital signal through an A/D conversion inside the thermometer.

• (3) The signal is then transmitted using a specific communication protocol.

The electrical signals generated by the vibration, rotation, and inclination angle sensors are amplified by the amplifying circuits and remain analog signals. Therefore, all the data except for the temperature data are digitalized by the PCI1718 data acquisition card. The PCI1718 data acquisition card meets the design and use requirements in terms of resolution, sampling frequency, accuracy, and number of input and output channels. The communication between the data acquisition card and the industrial control machine can be set up by itself, which is convenient to install with only plugging in the card and installing a driver. In addition, the data acquisition card also has a 4 KB data storage buffer, which is beneficial for high-speed continuous data acquisition. The performance of the PCI1718 board card meets the requirements for signal transmission. When input to the data acquisition card, the analog signal is converted into a digital signal and saved to the FIFO data storage area by the A/D converter. The interrupt trigger time needs to be set when used. Every fixed time, the program automatically extracts the data from the buffer to be processed. Generally, the interrupt trigger time is set shorter to facilitate timely data retrieval and prevent the original data loss due to data overflow.

Data communication

The system utilizes the RS485 Modbus protocol to communicate digitally between the instrument and host, data transmission and temperature control functions. The digital communication network utilized is the TIA/EIA-485 (RS485) standard. The wiring way is the two-wire system, and the protocol format is an 11-bit data format which conforms to Modbus-RTU standard: one start bit, 8 bits data bits LSB first, one parity bit (even parity) and one stop bit. On a single Modbus line, one master device (host) and one or more slave devices (sensors) are allowed. The data packets contain only the slave device’s address range of 1–247. The format of the data packet is as follows (Table 1):

Table 1 Communication packet format

The PCI1718 data acquisition card was added to the management system with underlying driver installation and header files added, with its RS485 communication protocol. As shown in the table below, the acquired data is transmitted to the system’s data processing module. Different relevant algorithms are used to process the data, including removal, filtering, integration, and signal compensation, to gain the desired feature values.

Among them, FCU represents the Field Control Unit, and FAU represents the Field Associated Unit. This paper utilizes sensors with different functions to detect the operational data of the converter during the smelting process, including inclination, rotation, vibration and temperature. The data collected by the sensors are transmitted to the industrial field processing and control unit through cables and fiber optics, and then the data are preprocessed and output to the DT system for calculation and analysis. Moreover, the signals of the converter inclination, oxygen lance height and oxygen lance flow are also connected to the system in order to integrate the signals input in the sensors to make a judgment of the steelmaking status. The DT system can also drive the sensors and field processing and control unit for data collection and auxiliary works such as water cooling and blowing.

Prediction of steelmaking endpoint and status monitoring

During the smelting process, it is possible to monitor various data such as temperature, vibration amplitude, inclination angle, rotation angle, input material weight, working time, and smelting time. By doing so, we can comprehensively understand the smelting status and predict the endpoint of steelmaking. Once we determine the ideal tapping temperature for a specific steel grade, we can perform timely tapping as soon as the corresponding temperature is reached.

To predict the endpoint of steelmaking, data such as rotation angle (R), material weight (W), smelting time (T), temperature (Temp), etc. is collected during the smelting process. By using this data, the time for the steelmaking endpoint can be predicted, and thus the timely tapping of steel can be achieved. ${W}_{s}$ in formula 3 is the standard input material weight required for different types of steel, ${T}_{a}$ is the average of each tapping time in history. $\alpha$, $\beta$, $\gamma$, $\varepsilon$ is the proportional coefficient. This article establishes the proportional coefficient of $\alpha = 0.1$, $\beta = 0.35$, $\gamma = 0.4$, $\varepsilon = 0.15$, which is based on the actual smelting process. This coefficient is most effective for predicting endpoint temperature.

$$ \begin{gathered} {\text{endpoint}} = \left( {\alpha \frac{R}{360} + \beta \frac{T}{18} + \gamma \frac{Temp}{{1200}} + \varepsilon \frac{W}{{W_{s} }}} \right)*T_{a} \hfill \\ \alpha + \beta + \gamma + \varepsilon = 1 \hfill \\ \end{gathered} $$

(3)

During the smelting process, monitor the vibration and inclination angle amplitudes. The system will prompt for abnormal converter status when amplitudes are too large.

When the predicted time for reaching the endpoint of steel production is reached, the steel can be produced. However, if the temperature cannot meet the required tapping temperature, the staff can choose from the following actions to remedy it:

• (1) If the temperature of the converter is too high before tapping, additional converter material can be added to the molten pool to cool it down. Point blowing can then be used to ensure the temperature and composition of the molten pool are consistent. Once the temperature has been measured and passed, the steel can be tapped.

• (2) If the tapping temperature is not much higher, clean scrap steel pieces can be added during external refining, or the blowing and cooling times can be extended.

• (3) To increase the temperature when it is too low, the furnace can have carbon added to it or silicon iron can be added for blowing and warming up. If the carbon content in the molten pool is high, it can be directly blown to raise the temperature.

When the converter status indicates an abnormality during smelting status monitoring, staff can quickly halt the process to ensure safety and prevent material waste.

System implementation and verification

Experimental environment and validation scheme

In order to facilitate the process management of the large-scale converter steelmaking process, this paper constructs an equipment and running status process management system based on the DT technology under a harsh environment, employing a three-dimensional model.

The field standard configuration of the DT-based process management system mainly consists of four parts, i.e., the System MCU, the FCU, the FAU, and various sensors. For example, the SA-S6016 dual-color infrared thermometer temperature sensor, the LC0403 piezoelectric accelerometer sensor, the MEMSIC company’s MXR7305VF tilt sensor, PCB company’s PCB 1102-05A rotation torque sensor, and the laser scanner Faro Scene scanner from FARO company were used in the experiment.

This paper presents the construction of a DT system on an 11th Gen Intel(R) Core (TM) i7-11700 K @ 3.60 GHz 3.60 GHz CPU with 32.0 GB of memory PC, Unity3D 2021.3, and Visual Studio 2013.

Figure 9 illustrates the experimental verification procedure. Initially, laser scanners and various functional sensors were employed to acquire smelting data in harsh environments. Subsequently, the 3D point cloud data obtained by the laser scanner at different angles was preprocessed and modeled for the subsequent scene construction of the accurate model of the converter. After that, the modeled point cloud data was reconstructed with a surface effect. Finally, the data obtained from different functional sensors was combined to implement the function of human–machine interaction. Compared with the direct design of the model, the modeling of the point cloud data is more conducive for users to directly observe the operation and usage status of the converter device.

Fig. 9

Experimental validation process

Validation process

Accurate device parameters facilitate user decisions relevant to the device. Prior to each converter refining task, personnel enter the steel refining system platform, and have to input the account password, data address and other information. Subsequently, they select either real-time or historical mode, as necessary, on the interface.

• (1) In the historical pattern, the post-refining task information of the converter, such as the lining thickness and lifespan, can be viewed. Real-time data is stored in a database. Users can recreate the state of corresponding time points based on the historical data, thus reproducing the previous refining process.

• (2) Upon entering real-time mode, all the sensors and corresponding transmission equipment synchronously enter the working state when the smelting task begins. The collected data is instantly transferred to the DT smelting process visualization system for analyzing and computing, and feedback to the model. The human–machine interaction panel displays real-time operation data including time, operation status parameters, and converter lining thickness map. Displayed on the right side are the weight of input materials, predicted endpoint time for steelmaking, forecast for furnace lining gunning, and monitoring data for converter operation status. The time displayed represents the current time of smelting, while the operating status parameter table includes information such as turning angle, temperature, converter life, and smelting time. Staff members can adjust the weight of input materials to obtain a more precise endpoint time for steelmaking.

Through experiments, the virtual ladle equipment model (as shown in Fig. 10) in the DT virtual smelting scene has achieved real-time synchronization and simulation with the actual equipment operating state (see Fig. 11).

Fig. 10

Model construction

Fig. 11

Operating scenes of the DT of a large converter steelmaking process

The inclusion of a view-switching module in the DT technology-based visualization and management system for steelmaking process equipment and operating state is presented under a harsh environment. By utilizing the 3-dimension and multi-view features of the virtual simulation platform UI interface, comprehensive visualization of the steelmaking movement state can be achieved. The module contains five controls including the main view, left view, top view, right view and the homepage. The corresponding angle of the steelmaking process model and parameters can be observed upon clicking a certain control. If any abnormal parameter is detected, proper steps can be taken to prevent safety accidents or waste of raw materials.

The 3-dimensional and multi-view features of the UI of the virtual simulation platform can achieve comprehensive visualization and display of the motion state of the converter smelting, and integrate them into the visualization interface by adding buttons, texts and controls, as well as changing their attribute values. In addition, using devices such as mouse and keyboard to modify the parameters of materials, colors, textures and maps, as well as adjusting the intensity and range of light source components to improve the appearance of the model, the visual degree of the DT scene can be improved. Finally, the static mapping from the virtual model to the physical entity is completed through external input events, thus realizing the static visualization monitoring of the converter steelmaking process equipment.

By clicking the view switch button, users can observe the multi angle modeling view of the real converter (see Fig. 12) in the system. With the clicked detailed data viewing button, they can view relevant information such as the internal converter wall and converter bottom thickness as shown in Fig. 13. The internal -180° to 180° range of the converter thickness result is unfolded. A different color is assigned to the converter lining thickness by the system and then segregated into relatively even grids. The mean value of the thickness is available in each grid, providing convenient observation for users. When the running state and lifetime of the converter exceed the established threshold range, the status indication light will turn red and an alarm will be triggered, reminding the users to take caution. In case of significant exceptions, such as large slope angles, an emergency stop command will be sent to the actual converter system. With such a DT system, users can make appropriate decisions based on anomaly changes during the operation. With the remote monitoring of the virtual model, the current smelting accuracy can be grasped.

Fig. 12

Actual processing scene

Fig. 13

Detailed data of converter view

A relevant database has been developed based on MongoDB-related software. Real-time data generated during the converter smelting process will be stored in a MongoDB database table. According to the thickness of the converter lining, the tilt angle, the rotation angle of the ear shaft, temperature, and time, the corresponding fields are established, respectively.

The presented virtual scene is of a 10:1 steelmaking converter production line, driven by real-time data from various sources. The virtual scene provides scalability, enabling observation of any corner of the converter production line. A chart can be called up by clicking on the converter device model to provide an understanding of the overall operation status and device parameters. Meanwhile, a DT system also reads data from an intermediary database to drive the mapping behaviors of the device model. The three-dimensional visualization of the live steelmaking converter process makes it easier to monitor, control and manage the process. Besides, a secondary development interface is reserved to allow interaction with Big Data and AI models to optimize the business.

Based on the data presented in Fig. 14, it can be inferred that the proposed DT system is capable of accurately predicting the endpoint temperature of steel during the smelting process. A comparison between the temperature values measured from the predicted endpoint time of different batches and the actual temperature values confirms the system’s ability to control the steel endpoint. As a result, this system can help reduce smelting costs and prevent material waste.

Fig. 14

Temperature monitoring data

Discussion

This paper presents an experiment conducted on a visual surveillance system for large-scale converter systems based on the homothetic scaling model in dual scenarios. Through the use of real-time data collection, transmission, processing, and storage of operational data in steelmaking processes, along with the mapping of visual management systems in digital twin scenarios, we have achieved intelligent control and dynamic data archiving of steelmaking procedures in harsh environments. The effectiveness of a DT process management system for poor environment steelmaking processes is verified.

This article proposes a system that can be applied to a steel plant to produce 20 MB of data per smelting, which includes all sensors except scanning. The scanner generates around 200 MB of 3D point cloud data per scan. In 2023, the monthly production capacity saw a significant increase as it reached 9365 tons. This is a 77.7% increase from the monthly production capacity of 5271 tons in 2012 when this system was not being used. The waste of smelting materials has also decreased from 34 to 7.8%, demonstrating a great improvement in smelting efficiency and reduction of smelting costs and waste.

Conclusion

In this paper, a DT-based process management system applicable to large-scale converter steelmaking processes is established in light of the low intelligence of the process and the unavailability of close monitoring by personnel under a relatively adverse environment. The converter smelting process is intelligently monitored, controlled, and interacts with humans using an established equipment model. Based on this, the key technologies of the DT system implemented in this paper are summarized as follows:

• (1) Improving the efficiency of the steelmaking industry with a DT-based management system: Integrating monitoring and forecasting functions, a process management system is established by combining DT and the smelting process. The system lets users visualize the equipment and operating status during the steelmaking process. By tapping at the predicted steelmaking endpoint in a timely manner, material waste can be avoided, and production efficiency can be improved. Provides an effective technology for automating steel smelting production.

• (2) Achieved higher fidelity digital display of the steelmaking process: An improved ICP algorithm for multi-viewpoint cloud data alignment and modeling is proposed. Abandon the rough digital twin modeling for high-fidelity 3D model construction using infrared scanners.

• (3) Realized multi-physical data analysis and prediction of equipment operation process status: The DT system integrates and analyzes physical information from multiple sources, and can predict the endpoint time of steelmaking while analyzing data. Realize digital analysis and prediction of information.

The construction of the above-mentioned system has realized the normal operation of the DT-based process management system for the converter steelmaking process equipment and operation status. Furthermore, the DT virtual scenario has validated the feasibility and effectiveness of the DT system in the converter steelmaking process under harsh environmental conditions. As of present, the framework system constructed has achieved real-time mapping from a physical system to a virtual system.

However, there are deficiencies in the feedback control of the physical system and more diversified data collection. This system demands equipment to be free of faults or possesses a low failure rate. It also requires accurate and reliable process data detection. The raw materials used should meet the standards of precision materials, possess stable quality, and require personnel of high quality. Therefore, promoting this system requires a certain amount of time and manpower costs. Moreover, the factors considered for predicting the endpoint of steelmaking in this system are not comprehensive enough. In the future, more factors such as steel composition, flame, oxygen content, etc. would be taken into account in the system to achieve more accurate predictions.