Robotic Systems in the European Steel Industry: State-of-Art and Use Cases

Abstract

The current digital and technological transformation affecting the European steel industry is carried out through the continuous and progressive application of Key Enabling Technologies (KETs) according to the Industry 4.0 paradigm. Among these technologies, robotic systems and autonomous vehicles (UAV/UGV), are increasingly applied in steel plants with the aim of improving health and safety in workplaces. Dangerous operations related to monitoring, maintenance and safety are often performed by using such systems in order to avoid workers’ exposure to accidents and injuries. Therefore, robotic applications and autonomous vehicles, which are suitably equipped with artificial vision and innovative sensors, are integrated in the daily operations by supporting humans and interacting with them according to well defined procedures, in order to minimise potential and dangerous events. The present chapter assesses the state-of-art for this technology, by analysing both the literature and European-funded projects, and investigates two specific use-cases related to different applications in steel plants with specific focus on the workforce impact in terms of implications in work organisation and operational procedure. Moreover, changed work patterns (e.g., behaviours, tasks, decision-making, training, skills, etc.) are also discussed considering human experience as a fundamental aspect.

1 Introduction

In recent years, industry in general, and the steel sector more specifically, are at the centre of a series of technical, social and economic changes under the flag of Industry 4.0 (Miśkiewicz and Wolniak 2020). Most of these changes involve the digitalisation of processes and products at different levels, possibly including the adoption of advanced Big Data and/or Artificial Intelligence-based tools, which can provide significant advantages in terms of productivity and socio-economic and environmental sustainability of steel production, with relevant benefits for workers’ health and safety (Colla 2022; Brandenburger et al. 2016; Branca et al. 2020; Vannucci et al. 2022; Colla et al. 2021a, b). However, for the complete integration of such technologies in the plant ecosystem and for a successful exploitation of new tools, some effort is required for managers and workers to update their skills and broadening their expertise (Branca et al. 2022).

In this context, robotics is one of the most promising technologies to be applied on the steelworks, characterised by the potential to improve both the productivity of plants and the safety conditions of the workers. In addition, robots’ versatility allows their employees’ involvement in diverse tasks that leverage their different characteristics. According to an estimation by The Boston Consulting Group (Ringel et al. 2015), the greater use of robotics and computerisation will reduce the number of jobs in assembly and production, but the creation of new jobs, particularly in IT and data science, will increase. The ‘Re-finding Industry Report’ of the European Commission, it is stated that over 1.5 million net new jobs in industry have been created since 2013 and a growth of labour productivity of 2.7% per year on average since 2009, higher than both the United States and Korea (0.7% and 2.3% respectively).

Robots are already used to perform specific tasks at various stages of the production pipeline. Many of them are part of manufacturing and assembly lines and operate in clearly defined areas. Conventionally, they are fixed to plant installations and perform recurring work to increase repeatability and product quality or operations that require the application of force or precision that are out of the human capabilities. Robots are tasked with performing operations that are potentially dangerous to humans because of, for example, the environment in which such operations take place or the inherent risks of the operation to be performed. Such situations are very common in the steel production and using robots can help preserve the safety of workers. In these latter applications, the tough environment of the steel plant featuring high temperatures, dust, steam, and aggressive agents represents for industrial robots both an opportunity and an obstacle, requiring the use of special arrangement to survive and perform flawlessly the assigned tasks in a harsh environment.

In the last decade, besides these stationary robots that have been used in the industrial setting for many years, the breakthrough evolution of technologies related to artificial intelligence (AI) has enabled the development of a new generation of mobile robots, capable of performing monitoring and inspection tasks by moving autonomously within the steel plant. This class of robots includes both ground vehicles (Unmanned Ground Vehicles—UGVs) and aerial ones, typically drones (Unmanned Aerial Vehicles—UAVs). In this context, there is potential for UGVs and UAVs to find countless applications by replacing humans in monitoring operations, reducing health risks and the number of accidents. Monitoring in these cases is a popular task. It can be related to product quality, abating the time of such activity, which is often determined by the difficulties of moving within the plant or reaching machinery or parts of it. In addition, when linked to maintenance activities, such operations will reduce plant shutdowns and increase safety and plant throughput.

In this chapter, the state-of-the art regarding the application of stationary and mobile robotic systems in the steel sector is evaluated through an analysis of the relevant literature and projects in the European arena that exploit these technologies in the steelmaking environment by referring to two specific case studies for the identified macro-types of industrial robots. This paper highlights the opportunities, issues and challenges behind the use of robots in the steel sector, focusing not only on technical aspects but also on the human–robot interaction and on the changes requested to managers and operators to fully exploit these new technologies.

The paper is organised as follows: in Sect. 2 a survey of the main application of robots in steel production is presented through an analysis of the literature on the state-of-art and the main research projects for both stationary and mobile robots. In Sect. 3, two use cases of successful research projects where such technologies are exploited in real industrial contexts are presented and discussed. Finally, in Sect. 4, some final remarks on this theme are provided together with the future perspective about the interaction between humans and robots in the steel plants.

2 Use of Robotic Systems in the European Steel Industry

Robots appeared in steelworks at the end of last century. In these early applications they were stationary robots, typically mechanical arms, to which repetitive and well-defined tasks are assigned. Later, the development of technologies and the spread to other sectors of industry brought increasingly complex robots with a greater degree of autonomy to the steel industry as well. Recent years have seen a greater use of mobile robots with varying levels of autonomy not only for performing a specific task, but also in terms of freedom of movement within the plant. In this section, a review of the state-of-art regarding the use of robots in the steel industry is presented through an analysis of literature and projects in which they are involved. Given the two macro families of robots, this analysis is divided into two sub-sections, the first for stationary robots and the other for unmanned robots, both ground and aerial.

2.1 Use of Robotic Arms in the Steel Industry

Stationary robots have found wide use in the steel industry to replace human operators in dangerous work or to be performed in unsafe or unreachable locations. Part of this family is constituted by robotic arms with a variable number of degrees-of-freedom (DOF) (typically 6) and complexity, equipped with different end-effectors and different level of autonomy, depending on the characteristics of the task and on their complexity. In most of the applications the task to be pursued by robots is clearly defined and their autonomy in terms of control strategy is limited to the task and takes advantage from other sensory systems or sources of information (Gerstorfer et al. 2018).

In the light of this consideration, many applications of robots in steelmaking are related to the casting process where their use can reduce human operators’ exposure to high temperatures or to the contact with liquid steel. An exemplar application of a robotic system equipped with a single multi-purpose manipulator demonstrates how different operations can be performed on the casting floor dramatically limiting the human intervention (Demetika et al. 2014). The main functionalities of this system include the identification and opening of ladle nozzles, shroud manipulation and tundish powder management, resulting not only in a safer operation for human operators, but in a better quality of the cast product as well. Another application exploits a six DOF robotic system with an integrated vision system for inspection purposes. In this latter work, the performance of the developed robot is analysed throughout several months of standard production both from the technical point of view and according to the user-experience (Hansert et al. 2016). The centrality of robots in the casting process is demonstrated in several review papers (Meisel et al. 2014).

Robots are used also for liquid steel sampling at different stages of the steel manufacturing chain. A 3 DOF robot, in which all three joints move in revolute form, is used to sample liquid steel from the electric arc furnace (EAF)—a task that would be very hard for human operators, considering the sensitive conditions around the EAF (Soltani 2010). In a similar application, the robot can be supported by a vision system to measure the steel temperature in a ladle during the casting process at different moments of the casting (Bian et al. 2022).

Robots in steel plants are used also for performing tasks that are unfeasible for human operators such as material movement and transportation. For instance, a gripper for steel transportation is used which can move within the plant different types and sizes of steel materials using an advanced multi-purpose gripping-hand that adapts to the different shapes of the handled material (Biermann et al. 2022). Further, a two-arms 6 DOF robot is used for the steel-strapping process of coils. Using this kind of robot—apart from the clear advantages in terms of safety—allows the handling of coils of different dimension abating the processing time and speeding-up the whole manufacturing process and improves the user-experience because of its flexibility (Lee et al. 2012).

Recent trends identify human–robot collaboration as a very promising technological field of development for industrial applications, and steelmaking in particular. The advantage of this approach is clear and is based on the exploitation of the strengths of human operators and robotic systems, avoiding their respective limitations. In this paradigm, the robustness of robots is utilised, avoiding human exposure to risks and potentially dangerous situations. The robots’ precision, speed and ability to repeat well-defined operations will also be exploited. On the other hand, the knowledge peculiar to experienced human operators will be used to control the robots and take decisions in a real-time manner together with the ability to adapt to new situations.

Although systems based on human–robot interaction have been already implemented in different industrial fields (Dannapfel et al. 2018; Gopinath et al. 2021) with successful results, not only from the technical point of view but from the social one as well, reaching a high level of acceptability, applications of such approach in the steel sector are not yet familiar. In this context, it is worth mentioning the semi-automatic cooperative robot designed to assist the building of the steel converter wall proposed in Ryu et al. (2012). The robotic system can transport and position refractory bricks in cooperation with a human operator understanding the human operators’ intention through a force/torque sensor. The interaction in this case avoids for operators any fatigue from movement of the bricks, which can lead to painful musculoskeletal diseases and grants the required precision for the bricks correct positioning. In addition, refractory brick construction becomes standardised so that it reduces the manpower and hours currently took in the same processes.

2.2 Unmanned Ground and Aerial Vehicles

In recent years, steel plants have proved to be an interesting field of application for the use of unmanned robots, particularly for tasks concerning product quality control and process monitoring—including structural inspection—of the plant. Such tasks, given the high productivity that is required, are essential to promptly identify eventual machine failures and ensure the continuous operation of plants and the maintenance of safe conditions for operators, as well as a high level of product quality. Using autonomous robots, which can move with certain freedom within the plant, reduces the time and cost of inspections, which, in this case, do not require actual plant downtime. In addition, most times, such robots have dimensions and movement capabilities that allow them to reach areas that are difficult for human operators to explore or that pose inherent risks due to, for example, height or high temperatures.

Navigation systems make use of a heterogeneous sensor and leverage cameras, GPS data, LIDAR sensors (Bachrach et al. 2012). In addition, in the presence of more than one robot, such data and information are shared and progressively refined to have at a global level a clear picture of the environment in which they move. Vision systems have made great strides for object recognition because of the advancement of Deep Learning and in particular of Deep Neural Networks (DNN) for object recognition purposes (Ahmad and Rahimi 2022). This technology allows robots to promptly and efficiently analyse the scene as it is taken by the camera identifying the position and the type of all the objects in the environment. Finally, in the last years, several breakthrough algorithms have been developed to improve the trajectory planning capabilities of UAVs and UGVs. These algorithms are designed to work flawlessly on controllers with limited computational and power resources and often deployed on optimised firmware that allows the full exploitation of their capabilities (Herrera-Alarcón et al. 2022).

The above-described technological advancements led to the deployment of UAVs and UGVs in many practical applications and to a blooming of literature works that discuss their achievements and issues. The steel industry has many characteristics that make it appropriate for the use of unmanned vehicles.

First applications of drones and ground unmanned vehicles were related to agricultural applications because of the greater simplicity of the context (larger environments, fewer obstacles) (Turner 2010). The monitoring capabilities of drones were exploited for checking gas pipes status (Rathlev et al. 2012) and for the assessment of the integrity of a photovoltaic field in through a swarm of UAVs (Grimaccia et al. 2015), highlighting the aspects of collaboration among swarm drones and on the determination of suitable path planning strategies. Unmanned vehicles are used for the distribution of light parts (small tools) to human operators in an industrial manufacturing chain (Orgeira-Crespo et al. 2020). Structural assessment is the main topic of several literature works. For instance, UAVs are used to detect defects and the health status of steel structures through an integrated image analysis system. That work focuses on the predictive maintenance of the structures and the results show this approach is the best cost-effective and time-compressing solution for this purpose (Chen et al. 2019).

While the literature on the use of unmanned vehicles in the steel industry is still sparse, there is an emerging array of projects in which the technology is being profitably adopted. In the recent EU-RFCS project DroMoSPlan, drones are used for monitoring different parts of the plant and processes related to steelmaking. DroMoSPlan is described more in detail in section “The DroMoSPlan Project” as an exemplar application of UAVs in the steel production. In another RFCS project, RoboHarsh, both UAVs and UGVs are used for different inspection tasks within a steel plant. In this project, the tasks pursued by different unmanned vehicles are deeply integrated with the maintenance strategies of the plant where drones and ground vehicles are assigned to complete periodic and continuous inspection tasks assessing process status and phases, product quality and safety issues within the plant.

3 Use Cases

In this section, two EU projects are presented as use cases where robotics systems are used in specific industrial applications. The aim is to put into evidence their achievements and an analysis of their impact from the technical and social point of view. The two projects involve respectively the two classes of robots previously identified, namely the stationary and the unmanned ones (basically UAVs and UGVs).

3.1 The RoboHarsh Project

The RoboHarsh project is an EU-RFCS funded project that perfectly highlights the benefits of using a robotic system in a harsh environment like a steel shop (Colla et al. 2021a; b), and the advantages conveyed by the involvement of the end-users in the early stages of the design of the robotic solution, according to a social innovation model (Colla et al. 2017).

The solution developed in this project consistently reduces the need for human intervention in laborious works, and thus limits related health and safety risks. In particular, RoboHarsh addresses the problem of the maintenance of the ladle sliding gate during the continuous casting process. In this context, the ladle is a container where liquid steel is refined to obtain the desired chemical and physical characteristics. More in detail, the freshly produced pig iron undergoes a decarburisation process before the next manufacturing phases. Subsequently, the ladle is used for the transport of the steel to the continuous casting machine. At this point, liquid steel flows through a tap hole from the ladle to the mould and the casting station, where it solidifies into semi-finished products as billets or blooms.

The flow of liquid steel is regulated by a sliding gate on the ladle bottom, which is operated by a hydraulic cylinder. During the process, because of the solidification of the steel, the gate is subjected to wear and deposition of solid material, which makes necessary some periodic maintenance on the sliding gate. During maintenance, the ladle is transferred to a dedicated area where some skilled operators analyse the gate surface, clean it and eventually replace the refractory components. Cleaning requires the use of an oxygen lance and is usually performed by two operators, who assess the wear status of the refractory. Based on this visual analysis, operators decide whether to substitute the two refractory plates. Normally, this operation occurs after a few subsequent casts, e.g. 5 or 6. Plates replacements include the positioning of the new plates into the suitable location with high precision (some millimetres) and the application of an adhesive mortar layer to ensure a perfect fit.

This operation takes about 30 min and exposes operators to a temperature around 70 °C. In addition, it requires the use of force from the operators since the weight of each plate is about 20 kg. Overall, this whole task is quite complex since it requires both strength, precision (for plates placement) and experience for the assessment of the wear status of the sliding gate. In this framework a robotic system could be helpful for the completion of most cumbersome operations and for those that involve higher risks for human operators due to their exposure to harsh conditions and high temperatures. On the other hand, human operators are still necessary for operations that need precision and decision capabilities. The context is thus suitable for a cooperative human–robot environment where each agent (human or robotic) is on duty for the task that it can perform best. Based on this consideration, the maintenance operation was subdivided by technicians and plant personnel into elementary operations that were assigned to human operators or to the robotic system as depicted in Table 1.

Table 1 Sequence of operations to be pursued during a maintenance cycle associate to the operation mode (automatic or manual)

The operations listed above are pursued on the original maintenance station that was adapted to the use of a robot in cooperation with human operators. This adaption process, as well as the selection of the hardware components, was performed by plant operators and technicians in order to consider all the relevant aspects of the design and implementation. Environmental factors, including plant conditions in terms of temperatures, dusts and operator safety and wellness, were also accounted.

The main robotic part of the system is made up of an industrial ABB robotic arm with foundry protection with a 6 DOF manipulator having the handling capacity of 245 kg. The maintenance platform was slightly modified to host the robot arm and grant operators’ safety. Some changes to improve operators’ working conditions were performed also outside of the maintenance platform by using conveyor belts for the transportation of the refractory plates to the area and the handling of the oxygen lance. The workstation is depicted in Fig. 1.

Fig. 1

The RoboHarsh workstation including a detail of the robotic arm in operation

The pulpit of the workstation is hosted in an air-conditioned container. All the main operations related to the movement of the ladle and the gathering of refractory can be pursued by the operators from inside the container. In addition, through a window, operators can perform some operations, such as applying the mortar on the plates, without exiting the container and exposing to risks or harsh conditions.

During the maintenance cycle, the robot exploits different tools that are placed in an ad-hoc designed warehouse (see Fig. 2) and are used by the robot to handle the oxygen lance, to remove the material residues from the nozzle and to spray the graphite on the discharger head. Therefore, depending on the specific operation, which is being performed, and has been previously acknowledged by the operator, the robot picks, uses and releases the necessary tool(s).

Fig. 2

The tool warehouse of the RoboHarsh workstation

The robot leverages a vision system for inspection operations throughout different steps of the maintenance cycle. The vision system is encapsulated in a single protective container which incorporates a 2D vision camera IDS UI-5270CP-M-GL (P/N: AB02037) (1/1.8″ 3.45 μm 2056 × 1542 Pixel) and a 3D laser scanner. The vision system communicates to the plant information system through a WiFi network, which allows a satisfactory band and avoids cables. The camera and 3D laser operate together, in particular the laser enriches the image provided by the camera with a 3D cloud of points. Both the image and the 3D cloud are pre-processed and denoised by using a suitably customised commercial software. 3D points are clustered by using a distance-based algorithm to identify the different basic elements of the sliding gate and allow the robot arm to operate.

A lot of attention was paid to the design and development of the Human–Machine-Interface (HMI) (see some screenshots in Fig. 3) to provide a smooth yet full experience to the operators. Through the HMI the operators can set-up the maintenance cycle in detail and monitor all the operations. Different cameras provide a set of images of the whole maintenance area on different screens and some additional screens show the main information provided by the robot during its operation. Operators interact with the robot through the touch screen that, for instance, allows to ask for more detailed information on certain tasks.

Fig. 3

Exemplar screenshots of the HMI of the RoboHarsh workstation

During maintenance, the robot is mostly autonomous but asks for acknowledgements to perform specific operations, such as the replacement of the refractory. Similarly, a complete view of the sliding gate is provided to operators by the camera mounted on the robot, nevertheless operators can decide to manually inspect the area, stopping any robot activity and taking the control.

Within the RoboHarsh project, the achievements of the installation of the described robotic system within a real steel plant were assessed in a twofold manner: from a technical point of view, by measuring the success rate of robot operations, and from the point of view of the impact on workers safety and acceptance. From the technical side, the operations minutely described in Table 1 and assigned to the robot have been grouped into macro-tasks and the success rate of each of them was calculated as the ratio between the number of operations totally completed by the robot and the overall number of attempted operations. The system was tested during one year of standard production. Moreover, other performance indicators were accounted:

• the difference in terms of completion time between the fully manual and the fully automated operation,

• the per cent reduction of exposure of human operators to harsh conditions.

The assessment of the social impact of the developed robotic station was made by means of a survey and interviews targeting both the solution developers and the operators of the steelworks. The survey aims at assessing the health, cultural and motivational changes due to the technological enhancement of the maintenance process.

The success rate of the robot macro-operations is reported in Table 2. According to the HMI set-up, in case of partial or total unsuccessful completion of the operation by the robot, the operator can complete manually the task. This is an important aspect of the human–robot interaction, ensuring the quality of the performance and showing that human intervention has to be an integrated part of new technological solutions. In this light, the results presented in Table 2 have to be considered as the worst case, since only the full completion of the task by the robot is considered successful while, on many occasions, the robot was able to perform part of the job, reducing the effort required to the human operators.

Table 2 Success rate for the automated task within the RoboHarsh project use case (Colla et al. 2021a, b)

In terms of maintenance cycle completion, the use of the robot slightly extended the task duration since the interaction with the machine brought in some latencies. However, such time difference is about 5 min which is considered tolerable. On the other hand, the use of the robot drastically reduced the exposure of operators to high temperatures. In fact, in case of successful completion of all the automatic tasks, such exposure time is reduced by 80% while in the worst case by 50%.

The project was successful from the social point of view as well. One of the main outcomes of the survey is that all participants agree on the fact that robotic assistance increases health and safety by reducing physical and heavy weight activities, exposure to high temperatures, and hazardous situations. In detail, the exposure to such conditions is perceived to decrease from 67 to 25% during the task. In addition, the system is acknowledged to make the job more interesting and to increase the level of the workers’ satisfaction.

3.2 The DroMoSPlan Project

The DroMoSPlan project is another RFCS-EU project where robots are used within a steel plant environment. In this case, it focuses on unmanned aerial vehicles (drones) and ground charging vehicles to support the drones during certain activities. The project was significant since it allowed an assessment of the applicability of such technology in a harsh environment, where UAVs and UGVs have to complete complex tasks and face diverse issues. Drones are basically used for monitoring purposes. The potential advantages in using drones in this context are many and include the ability to easily and quickly reach distant zones of the plant, to inspect areas that cannot be reached or are dangerous to humans (e.g. due to height or high temperatures or harmful emissions). In addition, the use of robots may allow such tasks to be performed more frequently than it would normally be done by human operators.

Within the project, 5 different use cases were identified and addressed. These tasks were developed and tested at the different partners sites and are related to the following areas:

• maintenance: with a task pursuing the inspection of plant buildings’ (1) roofs and (2) chimneys,

• environment: through a system for the (3) monitoring and (4) detection of chemical leaks,

• safety: through a (5) video surveillance system based on the use of drones.¹

During the project, particular attention was put in the design of the drones in terms of hardware robustness in order to resist the steel-works harsh conditions. Drones are provided with a communication system that allows them to share information with operators, ground components and among them. The UAVs are equipped with Pixhawk flight controllers that currently represent the state-of-the-art among open-source autonomous vehicles control systems and can be used for both UAVs and UGVs. The Pixhawk can be expanded and connected to other sub-systems for the wireless communication. In particular, in this case the SiK radio module was selected which provides a wi-fi serial interface commonly used for the transmission of telemetry and commands between a vehicle and a ground station. The message exchange between vehicles includes information such as position with respect to a fixed reference system, velocity and acceleration, battery status and is used for coordination purposes. Besides standard equipment, the diversity of the addressed tasks requires the employ of different sensors (e.g. CO detectors, cameras, thermo-cameras) which are eventually mounted on the drones.

On the operator’s side, missions are created and monitored through a ‘mission management’ software that operates through a web interface and allows operators to plan each mission step by step, monitor in a real time manner its execution and collect any relevant data during the tests for future analysis. The main algorithms for mission monitoring and drone control are developed and embedded in the vehicle architecture through an open-source software (ROS—Robot Operating System, platforms libraries) and the Pixhawk flight controller with firmware of Ardupilot.

The problem of drone autonomy was considered and led to the development of two technologies to address the needs inherent to the distinct applications. Different technologies for recharge (contacts and wireless) and different solution for drone-station synchronisation (stationary and mobile platform) have been developed. The stationary solution uses the landing platform as a recharge station (see Fig. 4). The drone, once landed, automatically recharges its batteries by mean of electrical contacts and without the need to remove them. During the landing phase, the drone can exploit an infrared beacon in the middle of the platform for precision landings on the platform.

Fig. 4

The drone positioned in the stationary recharge station

An alternative solution to the stationary recharging station is represented by a mobile station that implements a wireless charging technology. This solution is supported by the communication between drone and station in order to synchronise the landing point when the drone battery level is getting low. With this synchronisation, the recharging process can take place in a wide area of the mission scenario thanks to the mobility of the station. The mobile charging station is constituted by a UGV that comprises the charging box. The wireless charging system is made of two coils: a transmitting coil, mounted on the autonomous rover and a receiving coil mounted on a landing skid of the UAV.

Several use cases were selected in order to demonstrate the effectiveness and feasibility of drones employed for monitoring tasks in steel plants. For this reason, the applications are varied and have distinctive characteristics and diverse issues. Use cases with main achievements and critical points are briefly described in the following.

3.2.1 Plant Surveillance

This use case concerns the use of one or more drones for the purpose of surveying specific areas of the plant, but was not tested because of regulations pertaining to surveillance of workers at work. The proposed task of the drone in this case was to move within a specific area and detect the presence of unauthorised persons. If so, it raises an alert to the attention of the security staff. While the system was not tested, the theoretical approach can be reported.

The mission was to take place by patrol flights at random time intervals within the area of interest. In Fig. 5 an exemplar patrol path is represented, together with the location of the charging base for the drone. During the patrol, the drone reaches in sequence a set of pre-defined waypoints suitably located by operators to monitor the area and minimise the risks of collisions with building and cables within the plant. Waypoints are specified via their absolute coordinates while planning the mission with the previously described software component. When a new waypoint is reached, the drone rotates 360° on itself to check for the presence of intruders. When no person is detected, it heads to the next point. Between two flights, the drone reaches its base point where it is recharged.

Fig. 5

Exemplar patrol path for the surveillance use case

For this use case, the drone is equipped with an infrared camera whose images are processed by an object recognition algorithm seeking humans. The algorithm used is the well known YOLO and is based on a Deep Neural Network. In case of alarm, security employees are informed and, depending on the images they receive from the UAV, they can decide which steps to take next.

3.2.2 Gas Pipes Inspection

Within this task a single drone is used to check for the presence of CO leaks around the gas pipes of the steel plant. The task involves a human operator that brings the drone to the starting position for the inspection and supports the complete phase of the inspection in compliance with the EU regulations that do not permit completely autonomous UAVs operations.

The drone is equipped with a CO sensor whose positioning on the vehicle was carefully determined during laboratory tests. With a constant flow rate of CO through a small opening, the landing gear beneath one of the rotors turned out to be the most efficient because of the redirected airstream. Since the gas pipes network extends throughout a large area of the plant, is distributed through different levels and due to the presence of structures, curves and other obstacles, a purely visual approach was developed for pipes recognition and tracking to favour the exploration and check of the pipes. These latter operations are performed via a customised segmentation algorithm. Then, once the pipe is detected, the drone moves along it at a constant distance (obstacles permitting). In the ROS platform a dedicated algorithm customised to image processing is devoted to gas pipe recognition. Additionally, increased functionality was implemented, such as distance measurement and localisation of the pipe segments. CO measurement is permanently active and warns the operators in case of exceeding threshold value.

In this task the operator controls the drone (EU and workplace regulations make this mandatory). For this purpose, a dedicated smartphone app—of which an exemplar screenshot is shown in Fig. 6—was developed to allow the operator to continuously monitor the drone behaviour and the outcome of the segmentation algorithm to be sure that pipes are correctly detected. In case of CO detection, the UAV rises an alarm and signals it to the operator.

Fig. 6

Graphical user interface for the CO leak gas pipes inspection use case

3.2.3 Roofs Inspection and Defects Identification for Maintenance

This use case is related to maintenance of building roofs within the steel plant. The evaluation of the condition of a roof concerns not only the current state of preservation and possible damage due to weather conditions, but also the estimation of ageing rate. Using a drone for this task has some advantages among which a significant reduction in risk as the human operator is prevented from climbing on top of the examined buildings and moving on top of them (Stroud and Weinel 2020; Stroud et al. 2021). In addition, a drone can perform such an examination from different distances so as to have a better view of the roof surface.

Within the project, the experimental activity was performed on the surface of a roof of 500 square meters. During one mission, the drone flies above the roof and collects several pictures of it (90 pictures for the 500 m² roof of the test) which are then reconstructed into a 2D image of the whole roof (orthophoto) by using the software WebOpenDroneMap, an open-source tool for generating maps, point clouds, terrain and 3D models from aerial images. A reconstruction performed by this tool is presented in Fig. 7.

Fig. 7

Reconstruction of the 2D roof image from the individual images taken by the drone during its flight

Operators can examine this reconstructed photo; however, the visual inspection of a single picture requires time. For this reason, an automatic classification system was developed. The classifier processes individual tiles of equal size taken from the 2D roof image. For the training of the system, a set of tiles was manually labelled according to 4 classes, reported in Fig. 8: Defect, No Defect, Edge of roof, No Roof. Further, in order to increase the number of training samples, data augmentation techniques, including tile flip, reverse, pan and rotation were adopted.

Fig. 8

Tiles representation of an examined roof associate to status classification

The model used to perform the classification task is a Convolutional Neural Network (CNN) with 2 convolutional layers and 1 dense layer fully connected. The input was the 64 × 64 pixel sized tile, and the output is made up of 4 nodes, one for each class. The model reached 98% of correct classification, proving a satisfactory result.

3.2.4 Technical and Social Assessment

The developed approaches have been tested during the project both at the level of individual subtasks (e.g. obstacle avoidance algorithms, chemical sensors, classification algorithms, landing) and at the level of the macro tasks (use cases, except use case 5) passing all the validation tests and achieving satisfactory results that for the most part support the adoption of such technologies within the steel plant, which was one of the main purposes of the DroMoSPlan project.

This technology has some potential to be successful in terms of social impact and acceptability, but the productivity gains are questionable (Stroud and Weinel 2020; Stroud et al. 2021). During the project, different interviews were conducted to assess the impact of the exploitation of unmanned vehicles technologies on staff in terms of potential benefits and risks to the person. The result of this survey shows that the use of drones is acknowledged to potentially reduce the risks associated with a range of work processes, such as accidents (falls or slips when working at heights indoors; intoxication/poisoning when inspecting gas pipes, etc.). Further, in some circumstances, drones are also considered as saving both time and costs as they allow for immediate access to remote areas that, without using drones, require special equipment. However, they also create a range of additional work and training needs, e.g. data analysis from the sensors, which may slow the analysis process and not lead to the anticipated gains. The risks of health and safety stemming from drone crashes in outdoor areas are perceived to be less likely and so overall improvement in health and safety should be realised (Stroud and Weinel 2020; Stroud et al. 2021).

4 Conclusions

The analysis of the contributions in the literature, the research projects and case studies described in this chapter, highlight how digital transformation is at the heart of changes in the European industry and the steel industry in particular. In this context, we see an increasing use of robotic systems due, among the others, to the development of supporting technologies such as AI and sensors.

Robots are increasingly integrated into production pipelines and are in charge of tasks of growing complexity. The industry currently uses both standard stationary robots and mobile ones with a certain level of autonomy in the movement and operation. The first of the two approaches is the one still most commonly used. In such applications, the aim is to have robots do heavy or dangerous work or work that requires exposure to uncomfortable environments. New emerging technologies are also driving the use of unmanned vehicles such as UAVs and UGVs, which have the potential to be autonomous—should EU and workplace regulations permit—and expand the range of applications with monitoring activities at various levels that also cover aspects of quality control and maintenance. These applications can be the key to increased productivity, quality and cost containment, as evidenced by the results of many research projects. In this scenario, an emerging theme is that of human–robot collaboration to exploit the best capabilities of both: accuracy, reasoning ability on the one hand; strength, ability to repeat a task repeatedly, ability to operate in hostile environments, on the other hand.

In the projects mentioned and case studies described, the use of robots has brought the potential for multiple benefits in terms of productivity especially in terms of safety and risk reduction. From a social point of view, the use of robotic systems is also a success: because the robot is seen not as a competitor but as a collaborator who can help decrease the fatigue load and risks to which operators are subjected—it was not foreseen that the robots would replace workers in their tasks.

In the future, it is foreseeable that there will be considerable growth in using robotic systems, both stationary and mobile, in the steel context given the characteristics of the plants and processes. This growth will be driven by the unstoppable development of associated technologies and the increasing competitiveness of the steel market, which requires continuous technological improvements, without neglecting the necessary safety aspects.