Keywords

1 Introduction

XAI holds great potential for enhancing operators in modern factories, revolutionizing the way they interact with AI systems and improving their decision-making capabilities. By providing understandable and transparent explanations for the outputs of AI models, XAI empowers operators to gain deeper insights into the underlying processes and logic behind AI-driven recommendations or predictions. This enables operators to make informed judgments, validate AI-generated suggestions, and identify potential errors or biases in the system’s output. With XAI, operators can trust AI systems as reliable assistants, leveraging their expertise while retaining control and accountability. The ability to understand and interpret AI’s reasoning fosters collaboration between operators and AI systems, leading to more effective problem-solving, optimized processes, and improved overall factory performance. XAI serves as a valuable tool in bridging the gap between human operators and intelligent machines, driving a symbiotic relationship that maximizes efficiency and productivity in modern factory settings.

Potential applications of XAI in product optimization are as follows [1]:

  1. 1.

    Feature Importance Analysis: XAI techniques can identify the most influential features or factors affecting product performance, allowing businesses to prioritize and optimize those aspects for enhanced product quality.

  2. 2.

    Failure Analysis and Predictive Maintenance: XAI techniques aid in identifying potential failure modes, understanding failure causes, and predicting maintenance needs. This enables operators to proactively address maintenance issues, reducing downtime and optimizing product performance.

  3. 3.

    Continuous Improvement and Iterative Optimization: XAI enables operators to continuously monitor and evaluate product performance, identifying areas for improvement and iterative optimization. By understanding the factors contributing to product success or failure, operators can drive ongoing enhancements and innovation.

Potential applications of XAI in process optimization are as follows [2]:

  1. 1.

    Anomaly Detection and Root Cause Analysis: XAI techniques can identify anomalies in process data and provide interpretable explanations for their occurrence. This helps operators understand the underlying causes of process deviations and take corrective actions to optimize process performance.

  2. 2.

    Process Monitoring and Control: XAI enables operators to monitor and control complex processes by providing transparent explanations for system outputs and recommendations. This enhances operators’ understanding of the process dynamics, facilitating real-time decision-making and adjustments to optimize process efficiency.

  3. 3.

    Process Bottleneck Identification: XAI helps identify bottlenecks in complex process workflows by providing interpretable insights into the factors limiting throughput or efficiency. Operators can then focus on optimizing these bottlenecks to improve overall process performance and productivity.

  4. 4.

    Quality Control and Defect Prevention: XAI techniques can explain the relationship between process parameters and product quality. By understanding the key factors affecting quality, operators can adjust process settings and reduce the likelihood of defects, ensuring consistent and high-quality output.

These applications highlight the valuable role of XAI in process and product optimization, enabling operators to gain insights, make informed decisions, and drive continuous improvement in complex manufacturing processes. By enhancing transparency, trust, and collaboration between operators and AI systems, XAI paves the way for more efficient, cost-effective, and high-quality process operations.

This chapter begins describing CNH Industrial, which is a partner of H2020 XMANAI European research project [3]. CNH explores the use of XAI in a practical manufacturing context. This chapter provides a comprehensive overview of the use case and its application in today’s competitive business environment. It delves into the challenges faced in developing XAI within the current manufacturing scenario considering the maintenance operator’s needs during the XAI implementation.

Subsequently, the focus shifts to the XAI platform, briefly describing the scope of each component useful to avoid conventional “black boxes” AI approaches, due to their lack of transparency, introducing XAI techniques that brought a new level of interpretability to the field traducing it into “glass box.”

Furthermore, this chapter explores the explainability value and the methods to design XAI web-app considering user needs applied to a real-world case study where organizations have successfully employed XAI in their quality optimization processes. This case study highlights the tangible benefits derived from the utilization of explainable AI, such as improved anomaly detection and enhanced process efficiency.

Finally, the chapter concludes by discussing the XAI evaluation within the application on a real manufacturing environment for quality optimization and the Key Performance Indicator selected to measure the global benefits of the explainability. By the end of this chapter, readers will have gained some take aways of the role of explainable artificial intelligence in process and product quality optimization.

2 The CNH Industrial XMANAI Demonstrator

CNH Industrial is a world leader in the design and manufacture of machinery and services for agriculture and construction. It employs more than 40,000 people in 43 manufacturing plants and 40 research and development centers all over the world. The CNH Industrial use case for XMANAI European project focused on San Matteo plant, located in Modena, in Italy, where there is one of the most important research centers in the tractor field in Europe, using the most advanced technologies for design and engineering purposes. In addition to the San Matteo research site, Modena is also home to one of Italy’s manufacturing plants where the medium tractor transmissions are produced. They are used to assemble tractors in CNHi factories around the world. It is within this plant that the application cases of the European-funded XMANAI project originate.

The use cases focus on Modena Plant which is currently manufacturing 60,000 APL (all purpose low) and APH (all purpose high) tractors drivelines used to equip all tractors assembled in CNHi plants worldwide. The case study stems from a real problem that the Modena production plant frequently encounters nowadays. In fact, today’s production lines are mainly affected by unexpected failures of the production machine that stops the line for undefined periods of time.

Within CNHi’s production plant, the needs of the operators who face downtime issues every day lie in restoring the machinery, the beating heart of the plant, in such a quick timeframe that it does not cause major production losses. Downtimes are usually due to replacement of defective parts or for maintenance. XAI could act as a bridge between the machines and the operators, enabling them to understand in a quicker way the status of the machines and improving their productivity through a fast-responsive intervention.

The primary objectives CNHi aims to address through XAI within the XMANAI project, are ingest, manage and analyze the real-time and batch data acquired by CNHi systems, and the faults history data related to the maintenance and tooling systems to give user simplified suggestion thanks to explainability to restore the machine. The goal is to implement XAI models that should help the operator with recommendations, to optimize the production line avoiding waste of time and cost for the company.

In the next sections are described the use case and the current challenges the operators face during the use of machineries and how the proposed platform could enhance their work through the use of XAI.

2.1 Use Case Description

In the current state, within the Modena CNHi production plant, when a machine stops, maintenance operators must exclude different parts of the machine step by step to get to the faulty component and understand where the fault occurred and which anomaly caused the stoppages. As a result, operators waste significant amounts of time troubleshooting the faulty component and for replacing the component. Moreover, if the operator is not able to restore the machine, it is necessary to call for external maintenance operators, which slows down the process even more.

By implementing the XAI platform in the production process, the maintenance operators receive assistance in diagnosing machine errors. This support involves utilizing XAI suggestions derived from the sensor values installed on the machine. Thus, a twofold benefit is targeted:

  1. 1.

    Detect where the fault occurs so that maintenance operators will know which component is responsible and why, which allows to replace immediately the faulty part (for faster recovery).

  2. 2.

    Identifying the specific anomaly responsible for the failure assists operators in tracking the occurrence, minimizing troubleshooting duration, and enabling them to concentrate on pinpointing the root cause, thereby reducing recovery time.

To implement XAI within CNHi plant, the developed platform takes data from the current systems, carrying information about the status of the Heller 400, the CNC machine that is selected for the use cases (Fig. 1). A data pipeline will be created to ingest data and to train the XAI models and give suggestions from AI to the operator in an explainable form. In such a way, the user will test how to create a more organized data management and sharing and how to generate knowledge graphs with clear relationships between data and defined actions. Finally, the trained algorithm provides scheduling suggestions to the production manager to improve the scheduling in the Plant Management System (PMS) and visualization tools employed to aid operators by displaying and elucidating the AI results.

Fig. 1
A photograph presents a setup of the C N H i plant with the C N C machine.

The CNC machine selected for the use case

For the CNHi case study, explainability is indispensable to provide the worker with the necessary explanations and clarity to understand which part of the machine equipped with appropriate sensors may have caused the blockage. The worker must therefore be put in a position to decide and understand whether the artificial intelligence algorithm has correctly processed and considered the various possibilities of machine error and why it has arrived at that suggestion, according to which correlation between the various data extrapolated from the sensors. Several sensors were considered to equip the machine to be monitored by AI, but it could also occur that some machine malfunction could not be correlated with the currently installed sensors. Hence, it is crucial that the algorithm through explainability provides all the logical connections used by machine learning to make the operator aware of the machine’s operating/malfunctioning state.

2.2 XAI Technical Implementation

The data sources, collected by the sensors installed in the selected plant machine, a CNC work center machine named Heller 400, will interface with the advanced XAI platform developed specifically for the manufacturing industries, called XMANAI platform. The CNC work center is managed by machine conductor and shopfloor people in CNH. The machine is equipped with different sensors that collect data on the operating status of the machine and they are conned to the network. In detail, a dedicated management system is set up to collect sensor data (SmartObserver).

This use case focuses on anomaly detection on sensor data and explanation of machinery faults. The activities carried out for the development of the case study are discussed below, grouped in these different steps:

  • Data processing and data analytics: In this task, the complete history of sensor measurements was collected from the Heller 400 CNC machine. Data have been cleaned, sorted, and completed with the missing information, in order to obtain a usable dataset to feed to the selected ML models. Finally, the features selected have been identified as the complete set of 76 sensors data and a collection of recorded anomalies throughout the years.

  • Development of intelligent analytical model: In this task, a suitable set of ML models (Isolation Forest) was selected among others, taking into account the type and amount of data at disposal and the end goal of the detection task to be performed. For a more detailed explanation of the selected model, please refer to the upcoming section.

  • Training of the selected ML models: In this task, the selected Isolation Forest models were trained on a portion of the available dataset. The goal of the training was to correctly categorize a set of sensor configurations as anomalies or regular data. The set of models was then tested on the remaining portion of the dataset to generate insights about the model accuracy and quality of the predictions.

  • Definition of the explainability requirements and visualization tools: As part of this task, the explainability requirements were collected and analyzed to find the best output format in which the model explanation could be displayed, considering the target audience, their knowledge in term of explainability charts, and their explainability needs.

  • Production of visual explanations: The trained ML models are exploited to generate the needed explanation using a combination of custom tools and standard graphs, taken from the SHAP [4] library, and taking into account the requirements stated above. In this phase, the models produced explainability charts targeting individual sensor configurations (the one classified as anomalies) with local explanations, grouped sets of sensors (for multivariate explanations), as well as global explanations concerning the overall correlation between anomalies and the sensors having the higher contribution impact on these.

A high-level overview on the motivations behind the selection of an Isolation Forest approach as the basis for the models is provided in following paragraphs, along with the general idea behind the implementation of such technique in both univariate and multivariate cases. To this end, a crucial aspect that has been carefully considered for the current use case was the selection of the model to be used to accomplish the described tasks with the support of the XMANAI platform. The model needed to perform well with various irregular time series of varying lengths. It also needed to keep into account the possibility to add new sensors in the future without compromising accuracy or forcing to homogenize data every time. For all these reasons, an Isolation Forest approach was identified as the most suitable to use. This algorithm selects a feature and then randomly selects a split between minimum and maximum values of the selected parameter. The core idea is that many splits are required to isolate a normal point while a small number of splits are required to isolate an anomaly. The sequence of splits that brings to an individual data point is called path.

Depending on the path length, an anomaly score is computed and interpreted as follows:

  • A score close to 0.5 indicates a normal point.

  • A score close to −0.5 indicates an anomaly.

With this basic concept in mind, two approaches were explored:

  1. 1.

    Initially, the method involved fitting an Isolation Forest for each accessible sensor, followed by evaluating the models’ outcomes on the day when the failure occurred. Finally build plots to visualize and interpret the results. This method allows each individual model to be fitted on the specific sensor, increasing classification accuracy and making each prediction independent to the number of samples available for other sensors. This approach, however, prevents the user to catch correlation between sensors (since each model only refers to an individual sensor). This aspect is crucial in identifying possible points of failures in the machine, as multiple sensors can contribute to the same anomaly together. For this reason, a second approach was proposed to account for this scenario.

  2. 2.

    The second approach relies in fitting an Isolation Forest for each group of sensors and checking the results of the models in the day of occurrence of the failure. Groups were identified by domain experts considering the placement of each machine component with respect to the others and their reciprocal influence. Finally, some summary plots have been built exploiting the visualization capabilities of the SHAP library to interpret the results. The crucial aspect of this method is the selection of the groups; these should be large enough to allow the model to correlate together as many sensors as possible, while avoid grouping sensors with too different time series shape, so that only a few samples need to be discarded/extended to harmonize all sensors in the group.

The combination of the two approaches described allows the overall process to be precise enough, thanks to the individual models tailored for each sensor, while providing good correlation information, thanks to the categorization of the sensors. The visualizations produced by each of the two approaches are presented in the section below.

2.3 Explainability Value

Methods like questionnaires, user stories, user journey, and personas play a crucial role in exploiting XAI for product and process optimization. These methods facilitate a deeper understanding of user needs, preferences, and experiences, which are essential for designing and implementing effective XAI systems. The importance of these methods is expressed in more detail below:

  1. 1.

    Questionnaires: valuable tools for gathering quantitative and qualitative data from users. They help in capturing user perspectives, expectations, and feedback related to product or process optimization, collecting valuable insights that inform the development of XAI models.

  2. 2.

    Personas: represent fictional archetypes of target users, based on real user research. They provide a human-centered perspective, helping businesses empathize with and understand the needs and motivations of different user groups. Personas facilitate the creation of XAI systems that meet various user requirements, ensuring, in the case of CNHi, the optimization of the XAI web app interface for different maintenance profiles (Fig. 2).

  3. 3.

    User Stories: they provide a narrative description of users’ interactions with a product or process. They capture users’ goals, motivations, and pain points, highlighting key aspects that need to be considered for optimization. User stories also help in identifying specific areas where explainability is crucial to enhance user trust and decision-making. User story descriptions typically follow a simple template as a Card: As a < role>, I want <goal> so that <Benefit>. Finally, the key output from user story is a series of <Acceptance Criteria> preparatory to the interface design of the XAI web app (Fig. 3).

Fig. 2
3 illustrations present 3 figures of the maintenance operator, maintenance team leader, and maintenance technician with 3 respective text boxes. The text boxes indicate the title, tasks, and devices.

Personas developed for the CNHi use case

Fig. 3
3 illustrations present 3 figures of operators with 3 respective text boxes. 1, handle alarms and events with quick troubleshooting. 2, restore production by supporting in restarting the machine after failure. 3, monitor production in production control. The acceptance criteria are noted for each.

User stories developed for the CNHi use case

User Journey: maps to visualize the end-to-end experience of users throughout their interaction with a product or process. These maps illustrate touchpoints, pain points, and opportunities for optimization. Understanding the user journey helps in designing XAI systems that provide relevant explanations at the right moments, ensuring a seamless and trusted user experience.

Incorporating user perspectives through the use of these methods enhances the user-centric design of XAI, ensuring that explanations provided by AI systems align with user expectations, facilitate informed decision-making, and build trust. Ultimately, these methods contribute to the successful adoption and exploitation of XAI in optimizing products and processes to meet user needs and improve overall performance.

2.4 XMANAI Manufacturing App Experience

This section introduces the XMANAI platform key features and capabilities and discusses their usage in the CNH pilot. Subsequently, the chapter describes the manufacturing Web App developed to connect the generic platform functionalities to the specific final user to solve real manufacturing problems like troubleshooting.

2.4.1 XMANAI Platform and Components Usage

The XMANAI platform was developed to enable XAI takeup specifically for the manufacturing industry. It bridges the gap between the complex nature of AI algorithms and the demand for transparency, interpretability, and trust in decision-making processes within manufacturing operations.

Fig. 4 shows an overview of the components constituting the XMANAI cloud platform. In particular, the components surrounded in a red box are being employed in the context of the CNH demonstrator.

Fig. 4
A screenshot presents the components of the X M A N I A cloud platform. The various services highlighted are X A I secure sharing, insights, execution, pipeline lifecycle management, data manipulation, scalable storage, platform management, and data handler in data collection and governance.

XMANAI platform components usage

Starting with the XAI Insight Services and Data Manipulation Services, those are used at an early stage to perform tests on the models and the training data, in order to understand the challenges related to the use case realization (Fig. 5).

Fig. 5
A screenshot presents X A I pipeline designer page. The menus at the top are data import, experimentation, pipelines, asset management, and catalog. The back, edit, and control access buttons are to the right. The created, last updated details with result information and result columns are noted.

Visualization of the explanations through the XAI Pipeline Designer

The XAI Secure Sharing Services allows secure sharing of the input data between the CNH demonstrator and the technical supporting partners, granting CNH full ownership of their data in the platform and the ability to setup sharing policies according to company internal policies.

The XAI Execution Services allow to execute the XAI algorithms experimented earlier in a robust production-ready environment, once the models have been finalized.

Moving on to the XAI Pipeline Lifecycle Management Services, those are being used in all aspects of the process, in order to securely create, schedule, and monitor the execution of XAI pipelines (Figs. 6, 7 and 8).

Fig. 6
A screenshot presents the summary screen of the X A I Model Engineering Engine. 4 boxes below with titles, hyperparameters selected, validation technique, explainability tool selected, and performance metric list the details for the model selected.

Configuration of the model through the XAI Model Engineering Engine

Fig. 7
A screenshot presents the X M A N I A platform with the pipeline for the isolation forest. The pipeline menu is selected. The library, search field, output, and input are to the left, the design area is at the center, and the train models are to the right. Save and execute buttons are highlighted.

Design of the Isolation Forest Univariate pipeline through the XAI Pipeline Designer

Fig. 8
A screenshot presents the X M A N I A platform. The pipeline menu is selected. The logs are for the installation forest. The export result option is selected with a specific date and time. The details are indicated in the center.

Execution and monitoring of the Isolation Forest Univariate pipeline through the XMANAI Platform

The Scalable Storage Services are used both as the source of input data to be processed and to store the refined data undergoing the process of data ingestion. Those are further utilized to store the trained models and the associated prediction.

Moving to the Data Collection & Governance Services and, in particular, to the Data Handler category, the File Data Harvester component finds particular use to ingest the raw machinery data in the proper format, following the CNH data model and ensuring type consistency (Fig. 9).

Fig. 9
A screenshot of the home page of the X M A N I A platform. A title below, convert to X M A N I A data model with a table of 4 columns and 7 rows are indicated.

Dataset is imported into the platform through the File Data Harvester

Finally, the Platform Management Services are used to guarantee security and authorization of the users in the platform, allowing access to data and pipelines only to CNH and the supportive partners of choice.

It is important to highlight that, although some of the components haven’t been marked as used, they are exploited indirectly through other components of the XMANAI platform. The above description only refers to the components directly used in the realization of the use case.

2.4.2 XAI Powered Manufacturing Web App

The overall explainability design process resulted in the development of an interface for the manufacturing Web App that encompasses the explainability requirements and the acceptance criteria defined in the user stories. The manufacturing Web App, as an explainability tool, refers to a web-based application specifically designed to provide transparent and interpretable explanations for the outputs and decision-making processes of AI systems. It serves as a user interface that enables users to interact with AI models and gain insights into the reasoning behind AI-generated outcomes, coming from the XMANAI platform. The purpose of the Web App is to bridge the gap between the complexity of AI algorithms and the need for human understanding and trust. It provides users, such as domain experts, decision-makers, or end-users, with a user-friendly interface to access and interpret the explanations generated by AI models.

Through the Web App prototype, showed in Fig. 10, end-users can monitor information about the operating condition of the monitored machine. Looking closely at the different interfaces designed, in the Home screen there are information regarding the number of active sensors, the number of alarms and anomalies detected, together with the historical data of failures and a specific feature to compare trends of various sensors. The Sensor Values page displays the hierarchy of the most crucial sensors, utilizing a chromatic scale of red to indicate their level of criticality based on the algorithm confidence value. In the specific Sensor page, there is possibility to view its trend over time and compare different suggested and ranked anomalies. Finally, there is also a tab where the user can provide feedback on whether the platform’s suggestion was helpful in solving the fault or not.

Fig. 10
4 screenshots present web app pages of Machine X Y Z, sensor values, sensor Lorem Ipsum, and Lorem Ipsum problem. 1, 6 boxes with details are noted. 2, two dropdowns. 3, indicates limit and latest values, a dropdown of the last hour, a graph, and other details. 4, description, graph, and feedback 1.

(a) Home screen; (b) sensor values screen; (c, d) sensor details

The proposed manufacturing Web App prototype design was changed during the development phase due to technical issues related to the availability of data. The final developed manufacturing Web App has a simplified Home screen with just two widgets: Sensors values and Anomalies (Fig. 11a). The Sensor values menu allows the user to check the value of all sensors connected to the faults on different days, while the Anomalies menu includes the entire fault history of the machine. By clicking on Sensor values page (Fig. 11b), the application will show all the sensors in the selected time range, ranked according to the algorithm confidence value for the specific fault, highlighted by an increasingly bright red color showing the increasing criticality of the fault.

Fig. 11
2 screenshots present the web app pages of Heller 400 and sensor values with the algorithm's confidence value. 1, 2 boxes with details of sensor values and anomalies are noted. 2, rank values from high to low at a selected time, and explainability tools of X A I visualizations are noted.

Web App pages. (a) Home screen; (b) Sensor values screen

By clicking on the specific sensor (Fig. 12a), the Web App opens a page with the sensor’s trend where the user can check the value for each day and the potential anomalies (i.e., the top ten potential anomalies ranked by the algorithm) associated with that specific sensor. Displayed above the graph are the sensor threshold (if applicable) and the most recent measured value. In Fig. 12b, the graph shows suggested anomalies that can be associated with that specific sensor according to the value. The anomalies are ranked from the most probable according to the algorithm.

Fig. 12
2 screenshots of 2 web app pages. 1, sensor values present a line graph and details of suggested anomalies. 2, presents details of temporal explanations, trend explanations, and univariate explanations. A horizontal bar is noted below.

Web App pages for specific sensor. (a) Sensor’s trend; (b) XAI Visualization button

On the contrary, by clicking on the XAI Visualization button (see Fig. 12b), the Web App will open a page with all the XAI standard SHAP plots mainly designed for advanced users. Moreover, it is possible to select the type of explanation according to user expertise and role.

In addition to the Web App, an Augmented Reality (AR) application will be available to aid workers in executing maintenance procedures. It will provide step-by-step guidance for machine recovery through a suggested wizard interface. The AR app will assist the operator in solving maintenance problems and help the explainability to take place with the XAI algorithms.

2.4.3 AR App Design

Augmented reality (AR) combined with an XAI approach holds immense potential in supporting maintenance operators in manufacturing context. By overlaying digital information onto the real-world environment, AR provides operators with contextualized and visual guidance, enabling them to perform complex maintenance tasks more effectively. When coupled with XAI, AR becomes even more powerful by offering transparent explanations for AI-generated recommendations or insights, empowering operators to understand and trust the AI’s assistance. In the considered use case, the integration of AR and XAI benefits maintenance operators from different points of view:

  • Real-Time Visualization and Guidance: AR overlays digital information, such as step-by-step instructions, diagrams, or annotations, onto the physical equipment or machinery being maintained. This visual guidance helps operators locate components, identify potential issues, and follow the correct procedures. By combining XAI, operators can understand the reasoning behind AI-generated instructions, enhancing their confidence and ensuring accurate execution of maintenance tasks.

  • Predictive Maintenance and Anomaly Detection: XAI techniques can analyze real-time sensor data from the equipment to detect anomalies or potential failures. AR can then visualize this information, highlighting critical areas that require attention. By providing transparent explanations for the AI’s predictions or alerts, operators can understand the factors contributing to potential equipment failures, enabling them to take preventive actions or plan maintenance activities effectively.

  • Historical Data Analysis and Process Optimization: XAI techniques can analyze historical maintenance data, identifying patterns and correlations that are not easily noticeable by humans alone. AR can present this analysis visually, enabling operators to understand how past maintenance actions have affected equipment performance and reliability. By comprehending the insights provided by XAI, operators can make data-driven decisions, optimize maintenance processes, and improve overall equipment effectiveness.

The integration of AR and XAI in maintenance operations not only improves efficiency, accuracy, and safety but also empowers operators with explainable AI support. By visualizing information and providing transparent explanations, this approach enhances operators’ understanding, trust, and collaboration with AI systems, ultimately leading to optimized maintenance processes and increased equipment reliability.

From a practical point of view, the operator launches the AR application from the list of anomalies highlighted in the Web App. The AR application is running on a hand-held device like a smartphone or tablet, so the operator is able to intervene on the machine to restore the production through some digital instruction. Once the procedure is finished, the operator can give a feedback on the proposed workflow, in order to understand if it helped in restoring the machine.

2.4.4 Evaluation of XAI Platform

Definitely, the added value of XAI platform is the explainability associated with artificial intelligence. The evaluation of the platform through the questionnaire shows a positive impact of the XAI implemented in industrial environment. In fact, the ratings recorded during the questionnaire for the validation of the XAI Web App show that the explainability plays a key role (Fig. 13). Each parameter was rated by experts using a scale ranging from 1 to 5 (in which 1 is not at all and 5 is completely), obtaining a mean value of 3.83 (std dev = 0.83). For instance, when considering feature importance, a high value signifies that users can comprehend the significance of each feature in the final algorithm suggestion through the confidence score, expressed as a percentage in the output. And another example is the explanation clarity obtained by adding standard explainability graphs.

Fig. 13
A horizontal bar graph of questions for X A I validation versus preference values where 1 indicates not at all and 5 indicates completely. The bars of feature importance, explanation validity, and explanation clarity have the highest values. A vertical line at 3.83 is marked on the x-axis.

Questionnaire for XAI evaluation

The chosen Key Performance Indicators (KPIs) used to evaluate the advantages of integrating XAI into the production line primarily center around accuracy, reliability, and human-machine collaboration. These are outlined in detail in Table 1. Despite the current medium-high values displayed by the Key Performance Indicators (KPIs), there remains a necessity for additional optimization. This requirement arises from the fault grouping process, which was conducted solely by data scientists without input or support from maintenance experts. This optimization requires additional support from the maintenance staff to ensure greater accuracy and achieve the expected values.

Table 1 KPIs selected to measure the benefits of the implementation of XAI

Furthermore, samples of sensors’ data currently available are only based on few months of data coming from the new sensors (recently installed), but it is planned to expand the sample to optimize the predictions of the algorithm and achieve the expected levels of KPIs.

3 Conclusion and Lessons Learnt

During the development and integration of XAI modules for manufacturing tasks in the scope of the XMANAI, several issues have been addressed and overcome, which is summarized in Table 2.

Table 2 Lesson learnt table

At the technological level, an initial challenge was encountered in retrieving sensor data stored within the internal server to ensure real-time updates on the XAI web app. There should be an on-premises solution with a dedicated internal server with which the XAI Web App can easily communicate to update data in real time.

At pilot coordination level, there was a difficulty in collaborating constantly and effectively with the plant maintenance staff, who were often busy with emergencies to be solved on the production lines. Greater flexibility should be observed when assessing the availability of maintenance personnel, especially during the planning phase of implementation. It is essential to meticulously plan activities, considering the schedules and availability of maintenance workers.

The expected benefits of using the XMANAI platform will be the beginning of a human-machine collaboration in which the XAI will actively assist the maintenance technician during fault diagnosis. This collaboration will be achieved with a certain level of accuracy and trustworthiness of the XAI that will significantly increase the operators’ trust in XAI suggestions.