1 Introduction

Building on the foundation of Industry 4.0, Industry 5.0 represents the next phase of industrial evolution, emphasizing the seamless collaboration between humans and robots to achieve greater productivity, efficiency, and innovation [1]. It fosters cooperative work environments where humans and robots work side by side, interacting closely to accomplish tasks that play to their respective strengths. Robots can handle repetitive, dangerous, or physically demanding tasks, while humans focus on more creative and complex problem-solving activities. Advancements in artificial intelligence (AI) and robotics could enable robots to have a better understanding of human intentions and emotions. This would facilitate more natural interactions between humans and robots, allowing for improved teamwork and coordination. In Industry 5.0, we might witness the emergence of more versatile and adaptable robots capable of seamlessly transitioning between various tasks, working alongside humans in dynamic environments, thereby ushering in a future of continuous human–machine coevolution [2]. In Industry 5.0, the design of robotic systems would prioritize human needs, making them more user-friendly and easier to interact with. This would lead to higher levels of acceptance and integration of robots into the workforce [3]. It also means prioritizing the development of even safer and more ergonomic collaborative robotics applications to prevent accidents and ensure the well-being of human workers [4]. Human–robot collaboration (HRC) has the potential to bring about significant benefits for workers. By offloading dangerous and physically demanding tasks to robots, workers can avoid potential injuries and health risks associated with hazardous work environments. They can focus on more meaningful and intellectually stimulating activities, which can lead to higher job satisfaction [5]. Collaborating with robots can provide workers with opportunities for upskilling and learning new technologies, making them more adaptable to evolving job requirements [6]. Robots can complement human abilities by performing tasks with precision and speed. This collaboration can lead to increased overall efficiency in the workplace [7].

Collaborative robots, also known as cobots, are specifically designed to work alongside humans within a shared workspace or environment. In contrast to traditional industrial robots that usually operate within safety enclosures and execute repetitive tasks independently, cobots are engineered to directly collaborate with human operators, eliminating the necessity for extensive safety measures. Cobots are able to perform many manufacturing activities, such as pick and place, assembly, machine serving, quality inspection, and palletizing. They can handle repetitive or physically demanding tasks, while human workers focus on more complex activities, increasing production rates and product quality, as well as reducing physical strain on workers. Moreover, cobots can be deployed in hazardous environments to perform dangerous tasks that would otherwise put human workers at risk. This collaboration enhances worker safety and prevents potential accidents. Despite the increasingly wide range of cobot applications in manufacturing, their use in handcrafted manufacturing still presents several open challenges. The main objective of this article is to contribute to technological advancements in this less-explored field.

1.1 Cobots use in handcrafted manufacturing

Handcrafted manufacturing often involves intricate and artistic work where the human touch and creativity play a significant role. Cobots, by their nature, are programmed and automated systems, which may lack the nuanced decision-making and artistic judgment required in handcrafted processes. Handcrafted manufacturing often involves customizations, unique designs, and frequent changes in production requirements [8]. Cobots may struggle to adapt quickly to these changes without significant reprogramming or reconfiguration. Craftsmanship often requires precise manual dexterity and sensory perception, such as the ability to feel texture, gauge pressure, or detect subtle variations. Cobots may not possess the same level of sensory capabilities or fine motor skills as human artisans, which can limit their effectiveness in certain handcrafted processes [9]. Artisanal production can involve complex tasks that require problem-solving, improvisation, and decision-making based on contextual information. Cobots, while capable of learning and adapting, may struggle with non-routine tasks or unexpected variations that go beyond their pre-programmed capabilities [10].

In this context, collaboration between cobots and artisans is essential. Ensuring effective interaction, communication, and coordination between cobots and human artisans can be challenging, especially when the cobot needs to learn from and adapt to the artisan’s techniques or when real-time adjustments are required during the manufacturing process. Finding a balance between technological automation and preserving artisanal traditions is a delicate issue.

To overcome these challenges, it is essential to carefully consider the intricacies of handcrafted processes, foster close collaboration between artisans and engineers, and design cobot systems that complement and enhance artisans’ skills and expertise rather than replacing them entirely. This research aims to advance in this direction, focusing on the fashion footwear industry, where numerous processes still rely on manual craftsmanship. This reliance is primarily attributed to the industry’s substantial product variability and the essential need to maintain excellent quality and uniqueness. Typically, this sector, especially in the case of leather shoes, heavily relies on established, static technologies [11], with automation predominantly limited to non-artisanal activities (e.g., pick-and-place [12], gluing [13], and sole manipulation [14]). In this sector, cobots could be designed and programmed to perform tasks that require precise and repetitive movements, leaving the operator in charge of quality control, process supervision, and finalization. One task that meets these requirements in the leather footwear sector is, for example, polishing, which will be further explained in the next section.

1.2 Cobot-assisted polishing in manufacturing

Cobot-assisted polishing can be tailored to various manufacturing applications and materials. The specific tools, polishing techniques, and parameters may vary based on the requirements of the industry and the desired surface finish. Polishing HRC applications are commonly found in industries like automotive, aerospace, or consumer electronics, where metal parts require a high-quality surface finish [15,16,17,18]. Cobots equipped with polishing tools or abrasive pads can perform precise and consistent polishing motions, achieving smooth and shiny surfaces [19]. However, for components with intricate geometries, relying solely on a fully automated robotic method for polishing metal surfaces may prove inadequate, necessitating collaboration between humans and robots [20].

Although collaborative robots have found widespread use in polishing and other surface treatments (e.g., the glass industry, jewelry manufacturing, ceramic or porcelain products), and advancements in cobot-assisted polishing techniques continue to evolve [21], integrating cobots into the process of leather polishing can pose several challenges. Nevertheless, the general information on the state of the art in cobot-assisted polishing can be applied to leather shoes based on existing knowledge. Addressing these challenges requires a combination of advanced programming, sophisticated sensing and control systems, and a thorough understanding of the specific requirements for leather shoe polishing. Collaboration between cobot engineers, shoe manufacturers, and skilled artisans is crucial for designing and implementing cobot systems that can enhance the polishing process while preserving the quality and craftsmanship associated with handcrafted leather shoes. Table 1 summarizes the main challenges to address for employing cobots in the artisanal process of polishing leather shoes. They stem from both the thorough analysis of the literature cited in this work and the authors’ extensive knowledge of the process, acquired through longstanding collaboration with numerous companies in the footwear sector.

Table 1 Challenges related to cobot-assisted leather shoe polishing
Full size table

One of the very few works that have addressed these aspects is proposed by Borrel et al. [31]. The present article converges on many design choices of the robotic cell described in [31], but focuses on two aspects that need to be further investigated: the path generation, which Borrel et al. proved to be an impactful activity for the craftsman, and the process and the evaluations that lead to implementation in the real context supported by human-driven simulation. Finding the balance between productivity, quality, ensuring safety, and worker satisfaction is crucial for cost-effectiveness and meeting production targets. In this context, human-in-the-loop (HITL) plays a fundamental role.

1.3 Simulation and human-in-the-loop

It is important to note that successful HRC requires careful planning, training, and communication between workers and machines. Despite the high potential of cobots as an advanced future manufacturing technology, their practical implementation remains insufficient. An analysis of the success factors for their introduction has revealed that companies often underestimate employee-centered factors and struggle to find targeted solutions for integrating cobots into existing production systems [32].

Simulation and HITL techniques are valuable tools in designing and optimizing HRC scenarios. HITL refers to an iterative feedback process where a human interacts with algorithmically generated systems [33]. Researchers propose leveraging HITL to address challenges by incorporating human knowledge into modeling processes. This approach is utilized in simulation models that necessitate human participation or interaction, including models where humans are integral components of the simulation. Simulation and HITL enable iterative testing, evaluation, and refinement of HRC systems in a cost-effective and controlled manner, leading to more effective and successful implementation of HRC [34]. By utilizing simulations and incorporating human input through HITL evaluations, the design of HRC systems can be optimized for efficiency, safety, and usability [35].

Simulations allow designers to create virtual models of the cobot, the environment, and the tasks involved in HRC. These models can be validated against real-world data to ensure accuracy and fidelity. Simulations help in planning and optimizing the collaborative tasks performed by humans and cobots. They enable designers to analyze interactions, motion planning, and coordination between human operators and cobots in various scenarios, providing support for addressing the aforementioned challenges. However, current digital human modeling research is often limited to the development of static models with limited interaction [36]. By analyzing the simulation results, designers can identify feasibility concerns, potential bottlenecks, inefficiencies, or safety issues and refine the task planning accordingly. In handcrafted manufacturing, humans are a cornerstone. Therefore, simulations should incorporate considerations related to human factors, including ergonomic assessments and workload analysis, to evaluate the physical and cognitive demands imposed on human operators during HRC. This analysis helps in designing interactions and workflows that are comfortable, safe, and conducive to human performance. By virtually testing different scenarios, collision detection, and emergency stop mechanisms, designers can optimize safety measures and ensure that HRC systems meet safety standards and regulatory requirements. Human operators can be involved in the simulation process through HITL evaluation. This involves integrating human behavior models or using real human operators to interact with the simulated cobot in various scenarios. The feedback and observations from human operators can provide valuable insights into usability, human–robot interaction, and performance issues, guiding the design iterations. On the one hand, innovations in human behavior modeling enable more realistic simulations of human operators in HRC scenarios. On the other hand, immersive technologies provide a more natural and intuitive way for human operators to interact with virtual cobots and environments [37]. Integrating Virtual Reality (VR) and Augmented Reality (AR) into simulation platforms for HRC design, designers can observe human–robot interactions in simulated environments, evaluate ergonomics, and refine the design based on real-time feedback from human operators. Matsas et al. [38] demonstrate the effectiveness and acceptability of a virtual environment for assessing HRC, highlighting the need to improve tracking of user hands and physiological and performance metrics.

This article makes a significant contribution to both the scientific and industrial communities, and its key points can be summarized as follows:

  • Pioneering the study and development of collaborative robotics applications in the field of craftsmanship, a sector that has seen limited exploration.

  • Successfully implementing HRC in a real industrial setting, extending beyond the confines of laboratory experimentation.

  • Demonstrating the effectiveness of virtual simulation and HITL in achieving practical solutions for industrial environments while aligning with performance objectives of companies.

  • Prioritizing human factors in every stage of designing and developing HRC, with the active participation of artisans in the process.

2 From manual polishing to cobot-assisted polishing

The artisanal process of polishing leather shoes refers to one of the main Italian footwear brands and comprises three main phases (Fig. 1). Phase 1 acts as a preparatory stage and is relatively brief compared to the overall duration of the process. The role and expertise of humans are pivotal during this phase, as visual inspection entails setting crucial parameters for the subsequent steps. Indeed, visual inspection aims to identify aspects that could influence the polishing, such as the base color that affects the type and quantity of polish to be used, or any differences between the shoes in a pair in order to tune process parameters and standardize the final appearance. In the first phase, operators also insert a shoe last to ensure the correct resistance to the finger pressure.

Fig. 1
figure 1

Leather shoes polishing process

Full size image

The second phase involves the initial polishing using one or more polishes based on the shoe’s color. Operators utilize white tape for rubbing to prevent overapplication and ensure even distribution of polish. This step is crucial to prevent staining the shoe upon initial contact, which could affect the overall success of the process. The application of polish begins at the toe, counter, or lower edge, as these areas require more emphasis in achieving the desired shade, gradually progressing to other regions. Operators apply and manage the polish based on finger sensitivity.

In the third phase, similar tasks are performed, but with diluted polish.

In both polishing phases, the number of polish withdrawals and the application methods (movements and areas) vary based on feedback from visual inspection and the effectiveness of the process.

While Phase 2 and Phase 3 may appear extremely similar, they serve distinct purposes. Phase 2 focuses on crafting a uniform and durable base layer, requiring extended work time and greater physical exertion. In contrast, Phase 3 employs diluted polish to enhance shine, significantly influencing the final aesthetics and uniqueness of the product, albeit with reduced work time and physical strain.

The transformation of this manual process into a collaborative robotics application involves the cobot performing the second phase, while humans handle the other phases. To automate the second phase, standard polishing cycles consisting of a sequence of contiguous areas were established. The shoe is divided into distinct areas to define specific polishing rules, taking into account the type of movement (e.g., linear, circular), the number of passes, and applied pressure. Initially, humans set the right process parameters, and then, with human supervision, the cobot ensures a uniform, accurate, and consistent polishing layer upon which the craftsman can work later. Humans can customize the polishing cycles according to shoe characteristics, adjusting specific parameters (repetitions, speed, etc.) through the dedicated user interface. The craftsman ultimately completes the polishing process to guarantee uniqueness and achieve the desired aesthetic result.

Figure 1 provides an exemplifying process for confidentiality reasons.

The robotic cell is composed of:

  • A 6-DOF collaborative manipulator (UR5e) by Universal Robots with a built-in force/torque sensor.

  • A polishing tool.

  • An aluminum profile structure.

  • A workbench and protective barriers.

  • A rotatable shoe locking system.

  • An internal system to keep the inside of the shoe rigid.

  • A polish dispenser.

  • A system for the polish homogenization.

  • A laser scanner.

  • Programmable logic controller (PLC) and graphical user interface (GUI).

  • Electrical and pneumatic system.

The UR5e features six revolute joints and is designed for medium-capacity operations, supporting payloads of up to 5 kg and a reach of 850 mm. The cobot is securely fastened to the aluminum profile structure upside-down in order to reach all points of the shoe. The robust profile structure is engineered to withstand forces exceeding ten times the full torque of the cobot’s base joint and at least five times the weight of the cobot arm, maintaining stability without vibrations.

The polishing tool is designed to mimic the movement of the operator’s hand. It achieves this by rotating at 3000 rpm around its fixed vertical axis, creating a circular trajectory that replicates the motion of human fingers. It can be equipped with two backing pads, one with a 15-mm diameter and the other with a 30-mm diameter, allowing the robotic polisher to adapt to different parts of the shoe. The tool is always oriented orthogonally to the surface, and active force control is implemented using load cell feedback to maintain a constant 5 N force in the direction of the tool. Additionally, adaptive control of the tool speed is applied to ensure a constant speed overall, using the UR’s ‘moveP’ command, and reduce it in areas with high curvature.

The shoe-locking system is designed to securely hold the footwear in place, ensuring repeatability in the cobot-assisted polishing process and establishing a new reference frame located at the shoe’s toe. This system easily adapts to various footwear sizes thanks to its adjustable rear component. Additionally, the system can rotate the shoe to facilitate the execution of all polishing trajectories by the cobot.

The polish dispenser is engineered to dispense a predetermined quantity of polish, preventing both over-application and under-application that could negatively impact the polishing process. A homogenization system ensures the even distribution of polish on the polishing tool.

The robotic cell also incorporates an internal system to stabilize the inside of the shoe. Consequently, the footwear can withstand the forces applied by the cobot, minimizing local deformations of the shoe.

The PLC controls the cell components (i.e., cobot, polish dispenser, polishing tool, shoe-locking system rotation) and receives input from them to supervise the process.

Detailed technical specifications of the robotic cell systems are not disclosed due to confidentiality agreements.

The layout, which follows the principles of the golden zone and strike zone [39], is divided into three areas, as shown in Fig. 2, where the three phases are executed:

  • Phase 1 area is the storage zone where the operator inspects the shoes before polishing.

  • Phase 2 area consists of the cobot workspace, dedicated to the polishing of the shoes.

  • Phase 3 area is the manual processing zone, where the operator completes the final steps of the work.

Fig. 2
figure 2

Robotic cell layout

Full size image

Operator safety is ensured through a combination of protective barriers on one side and a laser scanner on the other. Access to the cobot’s work area is necessary for process supervision and parameter adjustment via the GUI. Sensors detect the operator’s presence and trigger either a slowdown (yellow zone) or a stop (red zone) of the cobot as needed. Normal manipulator speed resumes only when the operator exits the yellow zone. The polishing finalization by the operator (phase 3) occurs within a designated green area to avoid impacting the cobot’s productivity. The worktable is height-adjustable to accommodate operators with varying anthropometric characteristics, reducing the risk of postural issues.

Figure 3 shows the methodology followed for the design, development, testing, and validation of the cobot-assisted polishing process. Preliminary tests focused on the software selection and data workflow definition to implement the trajectories generation process. The cobot’s path is generated using NX Manufacturing software by Siemens (NX CAM), utilizing the “Multi-axis-deposition” processing feature. This allows for the creation and simulation of the tool path that deposits a layer of material on the selected surface of the 3D model. The output file generated by NX is then processed by RoboDK software, which generates the script file directly sent to the cobot controller.

Fig. 3
figure 3

Research and development methodology

Full size image

Compared to manual path recording, this approach has highlighted several benefits, as follows.

  • The use of CAD of the last, rather than of the shoe. The artisan footwear sector generally works in 2D and does not have a 3D model of footwear. The footwear reverse engineering process is time-consuming and still presents some unresolved critical issues. Moreover, the use of the last guarantees greater flexibility because it is used for different footwear models, drastically reducing the number of trajectories to produce. The difference between the last and the shoe upper is managed by the control system, which elaborates data provided by the force/torque sensor.

  • The path is automatically generated by the software based on some inputs provided by the operator (surface, direction, and distance between lines), which makes the process extremely streamlined and easy to learn. Trajectories can be modified by the operator to meet particular needs.

  • The path for the left shoe is automatically generated starting from that of the right, or vice versa.

  • The generation of the path for different numbers is semi-automatic starting from that of the sample number.

  • Times and errors during learning the path are drastically reduced.

Polishing tests consisted of three main sessions, each outlined in Fig. 4. In the first session, the leather was placed on a flat surface to evaluate, primarily, the elements of the robotic cell and the technical feasibility. This allowed focusing exclusively on varying certain parameters (speed, force, etc.), as the process was simplified with planar displacement. The leather appeared well-polished, and the polish was uniformly distributed on the surface, demonstrating its feasibility [40]. Controlling the amount of polish and its distribution was found to be a key aspect, requiring the redesign of the polish dispenser and homogenization system.

Fig. 4
figure 4

Polishing test sessions

Full size image

The second session was conducted on the shoe toe. This session involved studying control algorithms, testing various polishing tools, adjusting process parameters, and refining trajectories until achieving a polished shoe toe that met quality control standards. The curved surface of the shoe toe highlighted the need for implementing adaptive force and speed control. Position control proved to be impractical, as it often revealed a lack of contact between the tool and the leather due to slight differences between various shoe toes and their positioning, resulting in non-uniform polishing. However, the use of force control made it impossible to evaluate trajectories solely through simulation, as they did not perfectly adhere to the predefined coordinates. All shoe toe trajectories were implemented, optimized using NX CAM, and validated. Among the polishing tools tested, those made of expanded polyurethane performed the best.

The third session, conducted on the entire shoe, served to validate the shoe-locking system, optimize the shoe’s internal structure to withstand applied forces, and analyze the relative positioning between the cobot and the shoe to ensure reachability to all trajectory points. Once these objectives were met, leading to the implementation of a rotatable shoe-locking system and cobot reconfigurations, the optimal balance between polish quantity and withdrawals, cobot speed, rotation speed of the polishing tool, and leather processing time were investigated.

The optimization of the robotic cell and process parameters enabled the production of shoes to meet the required quality standards. Performance was also evaluated in the industrial environment, with quality control tests conducted by the company’s department on both semi-finished shoes produced by the cobot and the polished shoes finalized by the operator. The results obtained exceeded the current process, as the cobot can ensure higher quality and a more uniform base layer for final polishing, streamlining the operator’s finalization process. The direct involvement of operators in the design of the automated process, first in a virtual and then in a real environment, allowed for a collaborative definition of the parameters and trajectories that can be configured through the GUI for process control.

The critical issue encountered pertains to processing time and resource imbalance that hinders the integration of the robotic cell into the company’s production process. This problem frequently impacts the adoption of collaborative applications in real-world scenarios. Therefore, this article aims to harness the potential of HITL simulation to address this challenge. The next chapter provides a detailed description of this activity.

3 Simulation

To address the challenges that arose during the experimentation, various potential scenarios for the new work cell have been considered. The aim is to utilize HITL simulation to determine the optimal solution in terms of safety, ergonomics, and productivity. Two scenarios have been designed and subsequently simulated, both involving the deployment of two collaborative robots capable of working concurrently, with an operator overseeing their operations.

In the former, two cobots are concurrently processing two shoes at separate work cells, with the operator supervising both operations. The operator handles the loading and unloading of the workstations, monitors the entire inspection process, and adjusts parameters for shoe polishing. Notably, in this scenario, the cobots may not necessarily operate synchronously, offering several advantages as each cobot can undertake distinct tasks. For instance, one cobot can perform the standard polishing cycle, while the other can rework a previously polished shoe requiring additional processes. The robotic cell comprises two L-shaped conveyor belts, and the two shoe locking systems move in perfect synchronization, as illustrated in Fig. 5a. Specifically, when one locking system is positioned for processing (near the cobot), the other is stationed for loading/unloading and quality inspection at the conveyor belt’s end. After the cobot finishes the polishing task, the locking system is transitioned to the loading/unloading position, while the other one is brought into the processing position. This entire process unfolds seamlessly in a synchronized manner. In this way, when inspecting a shoe, the operator does not have to stop cobots, since he is out of the cobot’s working area. This setup fosters enhanced collaboration between the cobots and the human operator. However, the implementation of this solution does not entirely address the issues related to non-value-added times, primarily due to the necessity of reconfiguring the cobot and rotating the shoe to execute specific trajectories.

Fig. 5
figure 5

Work cell design scenarios

Full size image

On the other hand, the latter solution involves employing two cobots that collaborate on both shoes within a single work cell, minimizing the overall workspace requirements for the workstation. In this approach, each cobot performs specific trajectories on one shoe and different trajectories on the other. The cobot position is defined to facilitate access to specific areas of the two shoes without encountering kinematic singularities. As shown in Fig. 5b, the robotic cell consists of two cobots concurrently working on two shoes and four locking systems to position all four shoes. The locking systems on the same side are rotated 180° from each other, allowing the cobots to operate comfortably and avoid kinematic singularities. These locking systems are mounted on a rotary table, enabling the loading and unloading of shoes in an area situated away from the cobots. As a result, this configuration eliminates the need for reconfiguring the cobot or rotating the shoe. Additionally, the task sequence assigned to the two cobots has been carefully chosen to prevent any collisions between them or with the shoes. Four dispensers have been provided, one for each locking system, so that each cobot can use its dispenser without waiting for the other to finish the polish picking. Moreover, it can be useful when a dispenser runs out of polish during a job, as the task does not need to be interrupted to replenish the polish; instead, the cobot can pick the polish from the other dispenser. The defined configuration allows both cobots to access both dispensers. The operator performs loading and unloading activities simultaneously with the cobot tasks, as he is out of the cobot working area. However, unlike the first configuration, the two cobots must work synchronously on the two shoes, and there is no option to assign different tasks simultaneously. Specifically, it is possible to assign different tasks to the two cobots (e.g., assign only two tasks to rework the left shoe and perform a complete polishing cycle on the right shoe), but the two worked shoes can only be retrieved after the longer cycle is completed. In this way, there is no actual time-saving.

In both scenarios, the worktable and the GUI screen can be adjusted in height to accommodate operators with different anthropometric characteristics.

The workflow employed for the design and simulation of the robotic workstation is depicted in Fig. 6. The development and testing of the collaborative robotic cell involve a comprehensive workflow that leverages cutting-edge technology and virtual modeling. In the initial design phase, CAD systems are instrumental in crafting a detailed virtual representation of the robotic cell. This approach allows the design process, typically iterative in nature, to continuously evolve and improve through feedback and analysis. As described in the previous section, the generation of cobot trajectories relies on NX CAM, which specifically facilitates the creation of a polish deposition path customized to the unique shoe’s surface. Subsequently, the entire work cycle is simulated within Tecnomatix Process Simulate by Siemens, providing a comprehensive overview of both operator and cobot activities. Human actions are recorded using the Xsens MVN Awinda inertial motion capture system, while cobot motions are imported from NX CAM. Importing real human motion data, recorded with Xsens, into Siemens Tecnomatix Process Simulate enables authentic task analysis of humans performing real movements. By employing biomechanically precise digital human models, Tecnomatix allows for assessing work environments with virtual avatars powered by data from Xsens systems. This approach streamlines simulation development, simplifies setup, and enhances realism. Regarding the robotic movements, the cobot executes the polishing procedure by following the trajectories designed in NX CAM, which are further refined using the cls file containing a comprehensive list of all available points for the path. In Tecnomatix, these points are imported, with the z-axis oriented perpendicular to the shoe’s surface. It is crucial to maintain continuous contact between the Tool Center Point (TCP) of the cobot and the shoe’s surface to ensure that the z-axis remains consistently perpendicular to the surface. This phase assumes a pivotal role in identifying potential issues and refining the robotic cell’s performance. Advancing the simulation process, immersion into the VR realm is achieved by employing HTC Vive PRO, integrated with Tecnomatix Simulation and Xsens MVN Awinda system. This innovative approach transcends the boundaries of traditional testing, enabling users to actively participate in human activities within the virtual environment. This immersive experience offers invaluable insights into aspects such as ergonomics and workflow efficiency. Conducting tests within the VR environment involving multiple users provides a comprehensive evaluation of various critical aspects, including technical feasibility, productivity, ergonomics, and safety.

Fig. 6
figure 6

Design and simulation process workflow

Full size image

Both scenarios were designed using NX and then simulated in Tecnomatix Process Simulation software to assess their feasibility. The simulation involves a series of carefully coordinated actions performed by both the cobot and the operator. The polishing trajectories used in the two simulated scenarios are those validated with the robotic cell installed in the production plant (Fig. 4) to ensure the same level of quality.

In the first scenario, under full operational conditions, where the two cobots have just completed one shoe and are about to start another, the process unfolds as follows. Simultaneously, the cobot picks polish from the specific dispenser, while the two shoe-locking systems are repositioned to facilitate shoe loading and unloading. Subsequently, the cobot begins processing the new shoe while the operator assesses the quality of the shoe polished by the cobot. If additional polishing is needed, the operator leaves it on the locking system and configures supplementary trajectories through the GUI. In the case of satisfactory quality, the operator unloads the semi-finished shoe and places it on the work table. These steps are executed in the same manner for both stations, sequentially.

Following this, the operator selects the new shoes for polishing by meticulously inspecting them to determine the appropriate process parameters. The operator loads the shoe onto the locking system and selects the desired program for that shoe on the GUI, configuring the process parameters. This sequence of actions is then replicated at the other workstation. It is important to note that the operators can organize their activities as they prefer since these are independent of the cobot’s tasks. Consequently, they have the flexibility to choose whether to finish the second phase of manual polishing before inspecting the new shoe and loading it onto the locking system, or vice versa.

In the second scenario, under the same full operational conditions, the process is as follows. The rotary table is activated to position the new shoes within the cobots’ workspace, while the polished shoes are placed in the loading/unloading area. Subsequently, both cobots start the polishing operations on the two shoes simultaneously. In the meantime, the operator performs their tasks in a sequence of their choice, similar to the first scenario. The tasks include verifying the quality of the cobot’s polishing for both shoes, executing the second phase of manual polishing, selecting and inspecting two new shoes for polishing, positioning them on the locking systems, and configuring the parameters using the GUI. In contrast to the first scenario, the operator cannot allocate distinct programs to the two cobots for reworking a single shoe. Due to synchronized trajectories, involving both cobots is necessary, even if only one area of a shoe requires reworking, and initiating a new task in parallel is not feasible. Nevertheless, the time required for cobot reconfiguration, shoe rotation, and locking system handling is significantly reduced.

Additionally, in both scenarios, the operator is required to perform supplementary tasks, such as replacing the polish cartridge, changing the polish colors, and substituting the cotton lining of the polishing tool. In the first scenario, for cartridge replacement and color change, the operator can access the dispenser by opening a door on the protective barrier at the back of the cell, causing the cobot to stop. In both scenarios, changing the cotton lining can be performed from the front by placing the cobot in the service position.

Figure 7 illustrates the task allocation and the activity sequence for both scenarios, while Fig. 8 depicts the simulation of both robotic cells in Tecnomatix.

Fig. 7
figure 7

Task allocation and activity sequence

Full size image
Fig. 8
figure 8

Simulation of both scenarios by Tecnomatix Process Simulate

Full size image

Both scenarios were tested in a VR environment using Tecnomatix Process Simulate. The test involved two users with different anthropometric measurements (heights of 161 cm and 193 cm). The objective was to evaluate the proposed design of both workstations, identify errors, and improve the layout. It became apparent that the layout in both cases required refinement in terms of the positioning of various components, such as the shoe polishing table, cart with shoes to work, and cart for finished shoes. With the new layout suggested by the two users, it was possible to reduce the number of steps from 47 to 26 in the first scenario and from 30 to 18 in the second scenario. During the test, both operators benefited from the height adjustment of the worktable and the screen. Furthermore, safety conditions for the operator were verified by ensuring the accuracy of the areas defined for the laser scanner. The operators themselves stated that, during the simulation of various activities, they never perceived potentially dangerous conditions linked to collaboration with the cobots. This is attributed to the two layouts that ensure a well-organized workspace. This result is particularly significant considering that HRC often instills fear in workers and can introduce new stress factors.

Figure 9 shows the user, equipped with the HTC Vive PRO and 17 Xsens MTw (wireless motion tracker), unloading the shoe in scenario 1 and changing the polish cartridge in scenario 2.

Fig. 9
figure 9

Virtual reality simulation of both scenarios

Full size image

4 Results and discussion

The utilization of HITL simulation facilitated a comprehensive assessment across various critical dimensions. This involved a thorough evaluation of safety protocols, technical feasibility, productivity objectives, and ergonomic considerations.

The risk assessment was conducted in accordance with ISO/TS 15066:2016, which takes into account the combination of various risk factors, including:

  • The severity of potential consequences or effects stemming from a risk event (e.g., interactions between humans and cobots).

  • The nature of the contact (whether accidental or deliberate), its duration, and the human body parts involved.

  • The likelihood and frequency of a hazard occurring.

  • The potential for averting hazards or minimizing injuries.

Utilizing the Pilz Hazard Rating (PHR) technique, the level of risk can be determined through the following formula:

$$\mathrm{PHR }=\mathrm{ DPH }\times \mathrm{ PO }\times \mathrm{ PA }\times \mathrm{ FE}$$

Where:

  • DPH is the Degree of Possible Harm

  • PO is the Probability of Occurrence

  • PA is the Possibility of Avoidance

  • FE is the Frequency and/or Duration of Exposure.

Taking into consideration all the tasks assigned to the cobots, various potentially dangerous events were identified (Table 2). Based on the simulation of the new process and the safety systems implemented in the robotic cell, the identified risks have a negligible level of danger, as shown in the table. The laser scanners serve as continuous monitors and send signals to the cobot’s controller, enabling the implementation of programmed reactions if someone (either an operator or someone else) approaches the cobot’s workspace. Additionally, force control on the cobot enhances safety by enabling it to sense and respond to changes in force, thereby reducing the risk of injury in the event of contact with an operator or other objects. Furthermore, comprehensive information on safe operation is detailed in the user manual, including guidance on the use of Personal Protective Equipment (PPE). These guidelines provide operators with essential information about the specific boundaries within which the cobot can safely operate. Additionally, soft protections such as foam padding are applied to the cobot to add an extra layer of safety. These soft guards serve to mitigate impact forces in the event of accidental contact, minimizing the potential for injuries and fortifying the overall safety measures in the collaborative robotic cell.

Table 2 Risks identification
Full size table

As mentioned previously, despite a significant improvement in quality, the cobot takes longer to complete the first polishing phase. Additionally, various downtimes are observed for the cobot to allow the operator to carry out loading/unloading, quality control, polish cartridge change, etc. On the contrary, it takes the operator the same or less time to finalize the polishing compared to the as-is process, thanks to the better quality of the base layer. The delta time can, therefore, be utilized by the operator to perform other activities, including entirely polishing of special shoes (single model, particular materials, etc.) not suitable for the automated process.

Considering a fully operational condition, in an 8-h shift, tests in the industrial environment showed that HRC resulted in an approximate 20% increase in productivity compared to the manual process. The imbalance of the workstation, where the cobot is the bottleneck, has suggested new scenarios with two cobots managed by a single operator. From the simulation, both workstations were found to be perfectly balanced, and the delta time, drastically reduced, is used by the operator for all organization and supervision activities of the workstation without downtime. Without considering possible rework, the scenario with synchronous cobots allows a greater increase in productivity; however, assuming that approximately 10% of the footwear has to be reworked, this scenario becomes less efficient than the one with asynchronous cobots. As shown in Table 3, in both cases, productivity increases by more than 84%, and in scenario 1, it even reaches 88%.

Table 3 Performance comparison
Full size table

The ergonomic assessment was conducted based on the following standardized methods: RULA [41] and Checklist OCRA [42].

The biomechanical and postural load was objectively assessed using the XSens MVN Awinda motion capture system and the RULA method. The operator was equipped with 17 Xsens MTw to perform a full body analysis. Two complete cycles of polishing of different footwear were monitored. It is observed that in both cases, the operator is exposed to a medium–high risk, with a particular focus on the right side of the body. The primary risk factors are associated with the position of the wrist and neck, while no specific issues have been identified for other body regions. Another risk factor is related to the repetitive nature of movements involving steps 6 and 13 of the RULA. The introduction of collaborative robots, automating the initial phase of the process, allows for mitigating this specific risk factor. Specifically, most of the time, the risk transitions from medium (58%) in the current state (as-is) to low (68%) in the future state (to-be).

The OCRA checklist has been completed for both the as-is (manual polishing) and to-be processes. The duration of non-repetitive tasks, such as cleaning the workstation, replenishing lasts, and handling shoes, is approximately 15 min for a full shift. As depicted in Fig. 10a, the right side obtained a score surpassing the threshold, signifying a high-risk level, while the left side indicated a low-risk exposure. The overexposure to risk resulted from the highly repetitive nature of the polishing operation and the force perceived by operators. The force perceived by operators to execute the manual polishing phase was assessed, according to the Borg CR10 scale [43], as extremely strong since the initial phase of polishing generally requires more effort compared to the second phase. Based on the assumptions made for the new polishing process and the implementation of the robotic cell, the new OCRA checklist has been compiled. The two scenarios obtained similar results in terms of overall risk. As illustrated in Fig. 10b, the overall risk has been significantly reduced, shifting from very high to medium. The time for non-repetitive tasks has increased (approximately 90 min) as the operator must perform all auxiliary tasks, including loading and unloading shoes, changing the polish, and starting the PLC, resulting in a decrease in the final score. Similarly, the score related to stereotypy can be reduced for the same reason. Regarding the right side of the body, the time that the operator assumes an awkward posture (especially for the wrist) is lower compared to the as-is process, as the time spent on polishing has been reduced. Additionally, since the operator no longer has to perform the initial polishing phase, the perceived force required has shifted from intense to moderate. Artisans indicated that the first phase demands significantly more effort than the second. Moreover, the improved quality of polishing by the cobot has reduced the effort required by the operator during the second phase of polishing. Therefore, the introduction of the new collaborative work cell brings significant benefits in terms of reduced repetitiveness, improved ergonomics, and decreased force requirements for operators.

Fig. 10
figure 10

Checklist OCRA as-is (a) and to-be (b)

Full size image

5 Conclusions

This research, bridging the gap between craftsmanship and robotics, not only makes academic contributions but also provides actionable insights for industries seeking to embrace collaborative manufacturing. The study and development of collaborative robotics applications in the field of craftsmanship, a sector that has seen limited exploration, were successfully demonstrated in this paper. The implementation of cobot-assisted polishing of leather shoes in a real industrial setting addressed various technical challenges. Solutions were achieved through the design of dedicated components for the robotic cell, adaptive force, and speed control algorithms, process personalization autonomy for artisans, and the development of an efficient trajectory generation process easily modifiable by company engineers.

The success of the application was a result of a collaborative effort by a multidisciplinary team, including craftsmen, and extensive test sessions conducted in different environments: virtual, laboratory, and industrial. The first two environments primarily addressed technical, safety, and ergonomic issues, while the industrial tests focused on resolving organizational, production, and integration challenges within company workflows.

Quality control tests conducted by the company’s department demonstrated superior results compared to the current process. The cobot ensured a higher quality and a more uniform base layer for final polishing, streamlining the operator’s finalization process.

However, a critical challenge arose regarding processing time and resource imbalance, hindering the seamless integration of the robotic cell into the company’s production process. This article leveraged the potential of HITL simulation to address this issue. Two new scenarios involving two cobots managed by one operator were simulated in VR, showing a potential productivity increase of 88%. A key pillar of this study was the consideration of human factors throughout all phases of the process development. Safety and ergonomics were iteratively assessed involving operators, leading to the prevention of any potential safety risks and a significant reduction in work-related risks.

The utilization of VR and HITL was highly praised by users involved, thanks to its high fidelity and immersion. It facilitated the simulation of all flows and HRC, leading to an effective co-design of the workstation. This collaborative framework, with human feedback, holds the key to continuous refinement and optimization in the dynamic landscape of collaborative manufacturing processes.

Future work aims to explore the integration of advanced sensor-based and vision-based technologies to facilitate real-time verification and automatic adaptation of the polishing process, with the goal of minimizing rework and streamlining processing time. The simulation will be enhanced using controller-free hand-tracking systems to increase immersion and realism. Evaluation of the users’ cognitive sphere will also be conducted during the simulation.