Human-centered AI development in practice—insights from a multidisciplinary approach

Abstract

Industry 5.0 will focus on human-centered digitalization and put people at the center of manufacturing. To achieve this, it is necessary to consider employees throughout the transformation process. For this purpose, a practical approach to human-centered AI integration based on the Design Science Research method is presented in this article. The main requirements of the approach were evaluated in interviews with ten pilot companies, especially SMEs. The practical procedure was then examined in more detail for three application examples. The results show that there is still a lack of a uniform approach regarding the involvement of employees. Information on AI projects often reach those affected only during implementation. Even though the interdisciplinary approach is understood as a feasible approach for holistic AI development and implementation, there is still a lack of practical guidelines for adopting this multi-perspective approach. This work provides an initial starting point for a practice-oriented approach to the introduction of human-centered AI.

Practical Relevance: As a result of exponential digitalization and the associated in-depth use of partially to fully automated technologies such as AI, collaboration between humans and machines is becoming increasingly important. It is essential for companies to work according to human-centered approaches to consider motivation and satisfaction as well as aspects of data protection and personal protection for employees. The necessity for companies to adopt a human-centric and sustainable approach to digitalization will intensify in coming years. The work addresses the needs for operationalization of research findings for companies. Thus, the proposed guidelines can be understood as an approach to practical knowledge transfer from science to industry.

Zusammenfassung

Industrie 5.0 fokussiert die menschenzentrierte Digitalisierung, was den Menschen zunehmen in den Fokus der Produktionsprozesse stellt. Dazu ist es notwendig, den gesamten Transformationsprozess zu betrachten und die Rolle der Mitarbeitenden darin zu berücksichtigen. Der vorliegende Beitrag stellt einen praktischen Ansatz zur menschenzentrierten KI-Entwicklung auf Grundlage des Design Science Research vor. Die wesentlichen Anforderungen des menschzentrierten Ansatzes wurden aus Interviews mit zehn Pilotunternehmen, insbesondere KMU, abgeleitet. Anschließend wurde das praktische Vorgehen anhand von drei Anwendungsbeispielen im Detail analysiert. Die Ergebnisse zeigen, dass es an einheitlichen Vorgehensweisen bei der Einbindung der Beschäftigten mangelt. Informationen über KI-Projekte erreichen die Betroffenen oft erst während der Umsetzung. Auch wenn der interdisziplinäre Ansatz als gangbarer Weg für eine ganzheitliche KI-Entwicklung und -Implementierung verstanden wird, fehlt es noch an praktischen Anleitungen zur Umsetzung dieses multiperspektivischen Ansatzes. Die vorliegende Arbeit liefert einen ersten Ansatzpunkt für eine praxisorientierte Herangehensweise bei der Einführung von menschenzentrierter KI.

Praktische Relevanz: Mit der weiterhin zunehmenden Digitalisierung und dem damit verbundenen verstärkten Einsatz von teil- bis vollautomatisierten Technologien wie KI wird die Zusammenarbeit zwischen Mensch und Maschine immer wichtiger. Die menschzentrierte Gestaltung von Arbeit ist für Unternehmen essenziell, um Motivation und Zufriedenheit sowie Aspekte des Daten- und Persönlichkeitsschutzes der Mitarbeitenden zu berücksichtigen. Die Notwendigkeit, die Digitalisierung menschzentriert und nachhaltig zu gestalten, wird sich in den kommenden Jahren noch verstärken. Die Arbeit adressiert den Bedarf an Operationalisierung von Forschungsergebnissen für Unternehmen. Die vorgeschlagenen Leitlinien können somit als Beitrag für den praktischen Wissenstransfer von der Wissenschaft in die Wirtschaft verstanden werden.

1 Introduction

Digitalization has been characterized by exponential technological development in recent decades. The use of artificial intelligence (AI) systems, a term comprising intelligent assistance systems, machine learning technologies and other automated systems, provides companies with a significant competitive advantage (Fosso Wamba 2022). At the same time, these developments have a major impact on the members of the organization. Tasks that were previously carried out in analogue processes are increasingly being taken over by machines, which is significantly changing the way people work (Milanez 2023). Strategic and political measures are necessary to utilize the advantages of innovative technologies as effectively as possible without causing risks for employees. Therefore, a holistic and sustainable transformation approach is necessary to take all stakeholder perspectives into account. One point of reference is the position paper on the New Industrial Revolution, Industry 5.0, which was published by the European Commission in 2022 (Renda et al. 2021). Industry 5.0 focuses among other key factors on the human-centered introduction of artificial intelligence (AI) and puts people at the center of manufacturing (Khan et al. 2023). Although humans have already featured in approaches to Industry 4.0, the study by Hein-Pensel et al. (2023) shows that current methods and models still take little or no consideration of human factors. To achieve this, it will be necessary to consider the human perspective on AI through the entire process of digital transformation. This poses challenges for small and medium-sized enterprises (SMEs) in particular, as they often only have limited resources at their disposal (Woschke et al. 2017).

In addition, there are currently almost no standardized decision-making guidelines for the human-centric introduction of AI. Furthermore, there are hardly any use cases or empirical studies in which the successful use of human-centered design approaches or Human Computer Interaction (HCI) practices in SMEs have been examined (Brückner et al. 2023). Various studies (Hussain et al. 2018; Ullrich et al. 2023) have shown that involving employees from the beginning of the digitalization project and considering context-based suggestions is beneficial both to the organization and employees. For instance, this can contribute to greater motivation and understanding among employees when using the new technology. To achieve this, new guidelines and frameworks are necessary to guide a human-centered digital transformation in companies (European Commission. Directorate General for Research and Innovation 2021). In addition, when introducing and developing new AI systems, existing guidelines must be observed in all cases (High-Level Expert Group on Artificial Intelligence 2019). An increasing importance of trustworthy and ethical AI also highlights the fact that there are various principles which are often too abstract to be translated into concrete steps for the company (Prem 2023).

A systematic literature review of human-centered HCI (Brückner et al. 2023) showed that there is a general lack of practical use cases that demonstrate steps and concrete methods for human-centric digitalization. The study also revealed that among the minority of methods that describe specific use cases for HCI design, most solely focus on the technical implementation, while flexibility, cost-benefit ratio and suitability for companies are not considered. However, these factors are crucial to ensure applicability, especially for SMEs.

The paper therefore focuses on the research question (RQ):

“How can human-centered design and development of technical innovations, especially AI technologies, be carried out in a practical and scientifically sound manner?”

Based on a definition of human-centeredness, it is examined at which points in the development process employees should be involved. A central premise of this study is not to consider employees as one unit, distinguishing between workers, team leaders and managers, but to differentiate the stakeholder groups according to their points of contact with AI. By considering stakeholders and their perspectives, it is possible to assess which challenges may arise and which opportunities exist for successful implementation.

In this study, a holistic, stakeholder-oriented approach is employed. The term “stakeholder” carries many different meanings depending on the disciplines of use. Originally coined by Stewart et al. (1963), it was defined as “those groups without whose support the organization would cease to exist” (Slinger 1999; Freeman 2010 [1984]). Since then, hundreds of interpretations were given to the concept, each emphasizing a different set of attributes (Miles 2017). According to the definition of McGrath and Whitty (2017), which is in line with this current study’s context, here, a stakeholder is “an entity with a stake (interest) in the subject activity”. This definition is relatively broad and it is particularly interesting, on the one hand, to classify who would use the AI system and who not and who would be directly affected by the system on the other hand.

The stakeholder-oriented approach of this study integrates the relevant perspectives in the development of the AI system of the use case. The problem and stakeholders associated with it are identified and their roles and requirements in the use-case are analyzed. In the human-centered approach, the definition of stakeholders and the understanding of their participatory role is crucial, as they are actively involved in prioritizing the problems, finding the most relevant ones, and co-creating the solution. The stakeholders are involved via frequent communication and workshops. The solution development is carried out in an agile and iterative fashion: iterative cycles of planning, implementation, execution, and review are carried out with regular feedback to ensure that the project priorities remain aligned with those of the stakeholders. Recent studies which handle related topics are in line with the approach presented here (Humpert et al. 2023; Mundt et al. 2023).

The presented findings are the result of work in a multi-disciplinary research project, in which a total of ten pilot partners are being supported by an interdisciplinary consortium in the human-centered introduction of AI. In this paper, the focus is on three pilot projects within three companies. Two are manufacturing companies with different sizes, the smaller one being a family-owned business; the third company is a service provider.

2 Human-centered AI development

To answer the RQ, it is necessary to clarify what human-centeredness is. In scientific literature, human-centeredness and AI occur in various interconnected concepts that highlight different aspects of considering human needs, values, and capabilities in the context of AI-based technologies.

As a kind of an umbrella term, “human-centered AI” is used in different ways and due to its breadth, is, as Rezaev and Tregubova (2023) state, a “relatively empty” term. With the increasing popularity and significance of AI-based systems, the concept of human-centered AI has been used to emphasize the collaborative aspect and the co-creative process between an AI-based system as a supportive tool, and as an individual with the aim of enhancing or supporting human performance rather than replacing it (Ford et al. 2015). Human-centered AI design involves creating AI systems that align with human values and serve the needs of human stakeholders (Sun et al. 2022).

Even though the understanding of what human-centered AI is varies in detail, the debate about human-centered AI is a predominantly theoretical one that is shaped by the negotiation of the (im)balance between humans and technology.

In the concept of human-centered AI development, the focus lies on considering and embedding human needs and the human perspective throughout the development process of AI and beyond (Auernhammer 2020). In doing so, human-centeredness must not stop once an AI-based technology is trained and implemented but must consider the whole AI lifecycle and thereby ensures that the resulting technology matches certain principles for human-centered AI as explainability, transparency, ethical standards, fairness (e.g., non-discriminatory decision making), trustworthiness, responsibility and sustainability (Hartikainen et al. 2022).

When talking about human-centeredness in the AI development process, one must answer the question of perspective. When developing AI-based tools, it is not enough to solely focus on the needs of potential users but also to consider potential outcomes of an AI-based decision-making process. There are various concepts of differentiating stakeholders and reflecting on their roles in the context of AI adoption. Langer and Landers (2021) suggest dividing different parties in the complex network of different stakeholders when dealing with automation and AI: First, second and third parties. First party users are people who use the system themselves and work with its outcomes, e.g., by making decisions that affect other people. Those affected people, in turn, represent the second party. Often there is no consent or even knowledge of this party to be affected by AI systems. Third parties may feel that they could become a second party but are not directly affected yet. They understand and maybe fear the potential effect that AI-based systems might have on them and their own work. In many cases, in a company implementing AI-based technologies it is not obvious which stakeholders belong to which party. This three-party classification is useful to reflect on the question of how to integrate human needs and requirements.

Another approach is presented by Miller (2022), who distinguishes four stakeholder groups, namely non-stakeholders, who are part of the AI community and have an interest in AI development “but are not associated with or affected by the project”, development stakeholders, responsible for the technological realization of AI, usage stakeholders, who interact with the AI system, and external stakeholders like individuals or societal groups. External stakeholders can be compared to third party users in the Langer and Landers model.

What these exemplary approaches show is that, in the process of AI development and implementation, the range of stakeholder groups and individuals exceeds the mere distinction of user and non-users. A holistic human-centered approach must include those who might be affected and those having an interest in the specific technology or that might have to deal with the outcomes. The transition from the conceptual underpinnings of human-centeredness to practical implementation requires a structured methodological approach, which will be outlined in the following section.

3 Methodology

Main objective of the presented project work is the derivation of practical guidelines for the human-centered AI design and development. This is done by a critical analysis and reflection of a Design Science Research (DSR) approach for the prototyping of AI-based applications in manufacturing companies in West-Saxony regarding its human-centeredness.

The practical feasibility of this method is demonstrated on three AI-prototypes with three pilot companies as part of a research project that aims to implement AI technologies as means for human-centered and sustainable work design.

3.1 Research design for the analysis of human-centeredness in DSR

The research design that addresses our research question is shown in Fig. 1.

Fig. 1 Abb. 1

Research design

Forschungsdesign

In the first place, based on our parallel work on ten pilot projects, we derived research objectives which converge in the formulated research question. In the next step, three exemplary use cases were identified, taking their status, the comparability of the conducted process steps and their heterogeneity in company structure, size and digitalization level. After this, the documents and materials from all three pilot projects that were created during the first four DSR steps were collected and analyzed in detail. Furthermore, we systematized the presented AI systems and their relationships with the work systems across all companies, classifying them according to Niehues et al. (2023). In the result, the findings from the analysis were summarized and guidelines for a practical human-centered DSR approach were derived.

For each pilot, a systemic and interdisciplinary approach was taken, described in detail in the following chapter. Based on the comparison and analysis of these three approaches, practical guidelines, including guiding questions for the human-centered AI design and development for practitioners are deducted.

Each pilot partner was supported by a group of researchers from different scientific disciplines, namely “Ergonomics”, “Data Science”, “Factory Planning and Intralogistics” and “Service Engineering”, to enable a holistic view and design combined methods from all participating disciplines under the premise of human-centered AI development and implementation. In every pilot project, based on an initial problem identification, the development process of AI followed a DSR approach (Peffers et al. 2007) and common HCI guidelines (e.g., Pokorni et al. 2021; Merhar et al. 2019).

The DSR, with its systematic methodology, proves to be instrumental in addressing real-world problems and designing artifacts that align with user needs (Peffers et al. 2007). The DSR Methodology defines six steps to generate design knowledge:

• 1. Problem identification and motivation: In this first step, the problem is identified and defined which builds the basic precondition and justification for the development of a solution,

• 2. Definition of the objectives for a solution: In this second step, based on the problem definition, requirements are analyzed and objectives and resources for a potential artifact (solution) are defined,

• 3. Design and development: The work on the artifact, both conceptional design and development/construction happen in this third step,

• 4. Demonstration: Once the artifact is developed, it is used, e.g. “in experimentation, simulation, case study, proof, or other appropriate activity” (Peffers et al. 2007), to demonstrate its effect in contributing to the solution of the initial problem,

• 5. Evaluation: Through observation and measurement, the effects and efficiency of the artifact and its suitability for solving the problem is examined—the result of the evaluation leads to the decision for another iteration (going back to step 3) or ending of the design and development process, and

• 6. Communication: In DSR, communication is the final step that sums up the iterative design and development of an artifact and results in a scientific publication, presenting all phases of the design and development process.

In accordance with the current state of research activities, the first three DSR steps and methods that were applied during these steps are presented below. The DSR model includes communication as the concluding phase of the scientific process. In contrast, this study views scientific reflection and communication as an ongoing responsibility when pursued in the context of a human-centered approach. Researchers engaged in the DSR process should communicate with one another at every stage of the process, beginning with the problem identification and concluding with the reflection on the solution approach (artifact) to the results of the evaluation.

Furthermore, cooperation between different participants with mixed methods and across all DSR-steps supports the successful implementation of the approach. Research support activities, such as workshops and interviews, are crucial for iterative, interdisciplinary AI projects. This exchange should combine technical implementation, data science and the aspect of social sciences wherever possible (Parti and Szigeti 2021). Guided cooperation between scientists and the project coordinators of the companies facilitates data gathering and enables the validation of assumptions, understanding of user perspectives, identification of challenges and opportunities, and establishment of collaborative networks.

The accompanying scientific support addressed human-centeredness by strengthening the following aspects during the whole design and development process:

• Holistic view on stakeholders and sustainability (socially, economically and ecologically) of the intended solutions

• Transparent communication

Ethical aspects of AI (along the AI lifecycle—starting with the selection of datasets and potential bias to the impact of AI-based decisions).

3.2 Generalized workshop concepts

Besides the DSR steps, which in practice are specific for each company, special attention was paid to generalized formats, which were generally organized by scientific partner institutions and held together with all companies. Two workshop formats are presented in more detail here.

3.2.1 Workshop with focus group

The first phase of the research project involved recording the current status of participating employees in the different companies. The workshop was led by two scientists with expertise in the field of ergonomics and covered the following questions:

• Have employees been involved in the planning process of the AI project at this stage?

• When should employees be involved?

• Which employees should be involved?

• Why should certain employee groups be involved at this time? What are the hurdles and advantages in this context?

The workshop also included a presentation about the positive effects of involving employees in the planning phase of AI projects and how problems (e.g., in terms of fears) can be addressed. The aim was to raise awareness about employee participation among the project coordinators in the companies.

3.2.2 Data-workshop with experts

Furthermore, a workshop on different topics relating to data exploitation was offered. An expert in the field of data science imparted basic knowledge to the pilots. The requirements for this format emerged from the communication and needs of the pilot partners involved. It was addressing the following questions:

• Why does AI need data?

• What type of data does an AI-based system need and what types of training are there?

• What is machine-readable data and what is good-quality data?

• What examples are there?

This workshop left the participating pilots with an improved awareness about data management and quality and managed their expectations as to what AI can and cannot do.

4 Human-centered AI development in practice—the three pilot projects

The main goal of this paper is to present an approach for a human-centered design and development of AI, to reflect on the process of implementation and to assess the extent to which the procedure can be generalized and practical approaches for the human-centered approach can be derived. Therefore, to get started, the design and development process of three of the ten pilot partners is described in detail. The methods applied during the first three DSR steps as well as the respective application case and individual challenges are explained.

The pilot projects presented here were selected out of ten pilots because they are comparable in their process and progress but also state a certain degree of heterogeneity in terms of company size (number of employees), company type, organizational structure, and objectives. In Fig. 2 the relevant stakeholders and their role within the organizational structure are visualized. The following points are noteworthy: There is a coordinator (at least one) for the pilot-project in every company. In company 3, some functions are performed by one person, including the position of data protection supervisor whereas in company 1 this position is to the organization. In company 2, the fairness committee is comparable to a works council, but in company 3 there is no works council at all.

Fig. 2 Abb. 2

Organizational structure of pilot companies

Organisationsstruktur der Pilotunternehmen

4.1 Company 1

Company 1 is a service provider that works on different types of projects in cooperation with their customers. These can be long-time projects as well as short term commissions. Project acquisitions and requests go to different addresses, often in an unstructured format. Company 1 is divided into departments, each specializing in a particular field and spanning across two different sites.

The complexity and effort of the process of matching external service requests with suitable teams gave rise to the requirement for an AI supported competence matching. The aim of this pilot is to optimize service requests. The intended solution hereby supports team leads by analyzing historic data, identifying competences that were relevant in completed projects, and matches teams to incoming requests. The tool is to be used by the companies’ team leads. The team members themselves will not interact with the system but will be affected by the decisions made on its outcomes. The intended system is expected to bring benefits to customers (due to the improved speed in answering requests). In terms of the ethical and human-centered design of AI, it was important to consider possible negative effects at an early stage. The AI-supported competence matching at an individual level entails potential risks such as disadvantages for people with special competencies or people who are at the beginning of their professional careers. To avoid such negative effects, it was decided to assess competencies only at team level.

Table 1 shows what was done during the first three steps of the DSR process, how it was done, who was participating and what the objective of the participatory step was.

Table 1 Tab. 1 DSR process for company 1 (Step 1 Problem identification and motivation, Step 2 Objective definition, Step 3 Design and Development)DSR-Prozess, angewandt auf Unternehmen 1 (Schritt 1 Problemidentifikation und Motivation, Schritt 2 Zieldefinition, Schritt 3 Gestaltung und Entwicklung)

Individual challenges encountered in each company were addressed individually. In Company 1, for example, it is important not to be able to draw any conclusions about individual competencies. This is considered during the process by collecting competencies on team level only.

To create transparency during the process, the following communication activities were performed:

• Project was announced via information postings (intranet) to inform all employees about the pilot project.

• Roadmapping and project plans were used to visualize the process and time horizons for each phase for the internal usage of the pilot team.

• Regular (biweekly) meetings for project monitoring and between the pilot coordinator, the scientific team and the developing IT-team took place.

• The works council was informed about the pilot project, the used data sets and the suggested outcome of the AI-based system at an early stage (step 2, objective definition). It was decided that there is no further need for intense communication or collaboration between the works council and the pilot project team since no person-related data is included and the autonomy level of the intended system is low.

• Since it was decided by the project team to refrain from person-related data and determine competences only on a team-level, the data protection supervisor was only informed of the project and the trustworthiness and degree of the data-collection but not further involved.

4.2 Company 2

Company 2 is a supplier that produces automotive parts including sequencing from manufacturing to storage to assembly and shipping. The process is characterized by high product variance, changing order situation and a high degree of digitalization. Various workstations are set up in the factory building. Each workstation has different tasks and different ergonomic conditions. There are three shifts within a period of 24 h. Job rotation is already implemented, so that each worker has three different workstations in one shift. Currently, the shift supervisor determines the assignment of workers to workstations and the job rotation. The team composition of each shift is currently based on the experience of the supervisor, the skill level of the workers, the general risk assessment of each workstation and the planned produced output of the shift. In addition, the satisfaction of the employees is given high priority. In sum, this induces a high mental load to the shift supervisor to match all requirements. Altogether, this gave rise to the requirement for an AI based human-centred workforce planning.

To reach this, a staggered development is performed. The former analog team board was digitized and a feedback system, as well as a skill matrix was introduced for data gathering. These, in addition to historical data of production output, serve as input for the AI model. The digital team board shows a graphical overview of all workstations and assignment of employees by name. Currently, the shift supervisor uses it to assign the employees to each respective workstation. The goal is that the AI system provides recommendations regarding the shift compositions on the team board, to relieve the shift supervisor in determining optimal rotations. It will be possible for the supervisor to overwrite the recommendations. The shift supervisors will mainly interact directly with the AI. Although the shift personnel are being affected by the AI, they are not supposed to directly interact with it. Design and implementation were performed by the IT-team and coordinated by the project coordinator, neither of whom are directly affected nor have direct interaction with the AI.

The 3‑stage skill level matrix is used to define the composition of the team such that less experienced employees are working together with more experienced ones on a workstation. To capture the perceived team composition and perceived ergonomic factor of each rotation of each employee anonymously, a feedback system is installed in the factory building. Like feedback systems in supermarkets, the employee can click, pending the satisfaction on smileys from good to bad.

In conclusion, the focus in company 2 is on the development of an intelligent workforce planning system to contribute to a work environment that balances efficiency with employee well-being and physiological/ergonomic factors. The following variables were defined as the main driver for the proposed output of the AI system: perceived team composition and perceived physiological/ergonomic factors, desired produced output, and skill level of employees.

The following Table 2 illustrates, according to the DSR approach, the steps already performed to reach this goal.

Table 2 Tab. 2 DSR process for company 2 (Step 1 Problem identification and motivation, Step 2 Objective definition, Step 3 Design and Development)DSR-Prozess, angewandt auf Unternehmen 2 (Schritt 1 Problemidentifikation und Motivation, Schritt 2 Zieldefinition, Schritt 3 Gestaltung und Entwicklung)

To create transparency during the process, the following communication activities were performed:

• Presentation of the objectives of the whole project to the fairness committee (like works council) and some employees for initial information.

• Since it was decided by the project team to follow a trustworthy data-collection, the data protection supervisor was informed only. Nevertheless, to ensure respect for privacy and data protection, feedback loops over data collection, processing, and use were implemented with stakeholders. This avoids collecting data containing unnecessary personal or sensitive information.

• Internal Audits: Accompanying the introduction of the new feedback-system, the basic functionality and scope are presented to all employees.

• Concerning the new digital team board, personal direct communication between project coordinator, shift supervisor and IT team for iterative gathering of feedback take place. This leads to continuous evaluation and improvement, as well as better acceptance through joint design.

• There are regular meetings for project monitoring and updates between the project coordinator, the scientific team, and the IT-team. Usually via online meetings, but face-to-face meetings taking place as required.

4.3 Company 3

Company 3 is a small manufacturing firm which produces highly customized parts and components using CNC lathe machining. A major challenge in production planning is the quote preparation, which includes the cost and time for manufacturing the ordered products and must consider numerous parameters to adhere to the ever-increasing requirements for product quality. This process is labor-intensive and complex and must be rapid to guarantee the competitiveness of the company.

The use case here is an assistance system for the calculation of quotes: when an order arrives, the system shall first check whether similar orders have been produced based on image and text similarities between technical drawings of historical orders and the one provided by the client. Based on similar orders identified, the system shall predict the time and means required for manufacturing. The as-is status analysis showed that the whole process is manual and based on the experience of the director and the relevant employees and according to their multiple iterations of querying the ERP-system.

There are five people of various functions in the firm who currently perform the quote preparation, that is, they are the contributing stakeholders. Specifically, the directing manager devotes 50% of the time to quote preparation, the manager’s assistant 15% of the time, and the IT-person and technology experts about 10% of the time. These will be the ones who directly interact with the AI-tool and will benefit from the reduction of time needed to produce it. The production employees will not interact with the proposed system but may be indirectly affected by it, as accurate and informative quotes in a reduced preparation time can result in more orders. As an optimal outcome, it can help keep the company competitive, leading to increased job stability.

All activities performed during the first three steps of the DSR process are given in Table 3.

Table 3 Tab. 3 DSR process for company 3 (Step 1 Problem identification and motivation, Step 2 Objective definition, Step 3 Design and Development)DSR-Prozess, angewandt auf Unternehmen 3 (Schritt 1 Problemidentifikation und Motivation, Schritt 2 Zieldefinition, Schritt 3 Gestaltung und Entwicklung)

To create transparency during the process, the following communication activities were performed:

• Project announcement via information postings in the internal wiki system to inform contributing stakeholders about the pilot project. These have extensive knowledge about the processes and could contribute to process understanding.

• Participation of the directing managers and the manager’s assistant in workshops that introduce AI and data science to build trust in AI as would be implemented in the system.

• Regular meetings for project planning and monitoring between the pilot coordinators and the interdisciplinary project team.

• The director has also the role of data protection supervisor; the required data only involved information about orders and production and therefore any unnecessary personal or sensitive details have neither been collected digitally nor through interviews.

4.4 Clustering the AI-based work systems

The use cases in the examined companies address AI-support in competence management for personnel planning, project management and production planning. Table 4 lists the requirements, the chosen AI approach and data basis as well as the level of experience regarding digitalization and AI for each company.

Table 4 Tab. 4 Pilot companies and AI approach of the respective pilotPilotunternehmen und die KI-Ansätze des jeweiligen Pilotvorhabens

To systematize the AI systems and their impacts on work across all companies, the classification system developed by Niehues et al. (2023) is utilized. This heuristic includes six key features or attributes of AI within work systems—systems in which humans and/or machines perform processes and activities to produce a specific product for a specific internal or external client using technology and other resources (Alter 2013), see Fig. 3. The first feature within the classification is the AI function which relates to the means of work and task relationships. All pilots of the analyzed companies primarily manifest information acquisition and cognition functions. The second feature concerns AI usage within the work system, whether as a supplement to or a replacement for existing tools and/or human labor. In all three companies, existing tools and human work are being supplemented. The third feature is the Level of Autonomy, describing the degree of decision-making and control exercised by the AI ranging from recommendations provided by the AI-based system (level 1) to fully autonomous operations (level 5). Company 3 focuses on Level 3 autonomy, i.e., the system is granted limited autonomy and the human is prompted to confirm the system’s decision or act as a fallback level. The other companies are at Level 1, i.e., the system assists with selected functions, but the human is always responsible and makes all decisions. The fourth feature is the Visibility of the AI, which pertains as to how recognizable the AI is to workers. The companies implement visible systems—where users may recognize that they are working with the help of AI. Feature five is the Human-AI Relationship, which can be reciprocal—involving interactive elements between humans and AI, one-sided as a unidirectional information transfer, or coexisting as independent work systems. For the three analyzed companies, the human participants shall submit requests to the respective AI systems and obtain responses from them, thus the AI systems can be classified as interactive. Finally, the sixth feature is the interface modality, detailing how inputs and outputs are managed. Here, the companies utilize a visual interface with control elements.

Fig. 3 Abb. 3

Classification of pilot projects based in classification scheme for AI systems by Niehues et al. (2023)

Klassifikation der Pilotprojekte entsprechend des Schemas von Niehues et al. (2023)

To sum up, within the paper, early-stage AI systems (in Autonomy Level) with a supplement characteristic are addressed—mainly based on visual interactional components.

5 Findings of the human-centered approach

In this chapter, findings and experiences from the practical application of the DSR steps are described.

5.1 Process and method reflection

The methods used were essentially tailored to the individual requirements of the pilot partners and adapted iteratively during the process.

During step one of the DSR process, the problem identification and motivation, an initial semi-structured interview was conducted with all pilot partners. This allowed inferences about the status of companies in order to examine, among other things, the strategic suitability of AI for the individual company, which should be considered as the first step in the human-centered introduction of AI (Pokorni et al. 2021). Moreover, these interviews were used for the definition of clear goals for the pilot projects by collecting various perspectives on the potential of AI, ethical implications and individual needs (What is needed? Who needs it? What level of automation is targeted and what impact does it have on the different stakeholder groups?). As shown in Chap. 4, the stakeholders that participated in these initial interviews were mostly part of the management level. This results from the given funding project context in which the pilot projects are located. The effect of this top-level perspective is, that the addressed AI solution targets on planning and managing processes. However, by promoting bottom-up innovations, companies should encourage workers themselves to initiate optimization processes.

In the second DSR-step, the objective definition, both further interviews and workshops were used. First, the as-is status and potential changes following the initial interviews were discussed to draw profound conclusions on the potential impact of the solutions. The additional interviews were used to discuss and debate specific topics that could be derived from the first project phase. Depending on the company, these included, for example, ethical aspects like privacy and data protection, requirements analysis in relation to the use case and analysis of the database. It was noted that in cooperation with the largest of the companies analyzed (Company 2), workshops were previously used as a method in addition to interviews. In line with the requirements for human-centered AI development, additional stakeholders were integrated in this second phase. This integration of further perspectives enabled a holistic view on the objective outcome of the AI projects, their risks, and potential barriers to success. Workshops and interviews with employees were used for multi-perspective reflection on process errors and targeted brainstorming and identification of potential data basis and AI technology that suits the objective. The external perspective offered an unbiased view on processes and potential solutions. Moreover, it became evident that all three companies, despite their differences in size, level of digitalization and experience with AI projects, have benefited from, or were reliant on, external expertise regarding the suitability of AI-based technologies and the identification of suitable data sets and the process of data labeling.

In the third DSR-step, design and development, the methods used by the companies differed widely. On the one hand, this heterogeneity is a result of the different use cases and associated requirements. On the other hand, Company 1 and Company 3, for example, increasingly used workshops in the development phase, while Company 2 had already conducted these in an earlier project phase. Other methods used in step three were internal audits. These were conducted in Company 2 only, which also has the greatest number of employees. The design process of the AI projects revealed that the pilot companies needed support in selecting suitable methods and in deciding upon the relevant stakeholders to be involved in the design process. Although workshops are a common tool in science, this is not always a matter in practice.

Generalized workshops with all companies involved were performed. The conducted workshop with the practical target group was intended to encourage a holistic view on the projects by addressing the questions of when and how employees are involved in digitalization projects. All participants were asked the same structured questions (see Chap. 4). The answers showed that only one company communicated the AI project to all employee levels during the planning phase, however, in this case exclusively via indirect communication (Intranet). In the planning phase, none of the companies directly communicated with the affected employees (e.g., in the form of internal audits or presentations).

To ensure transparency about the pilot projects and the addressed changes of working processes and the impact on different stakeholder groups, two companies used digital methods (intranet and internal wiki) for communication in the further course of the project. Non-digital communication methods included direct communication between the involved actors, audits and workshops as well as visualizations of the project steps with a time horizon in the form of a roadmap.

Except for the company which chose to inform all employees right at the beginning of the project via intranet, the remaining companies initially rated communication at management level or project team level as sufficient for their AI projects. The main hurdle mentioned for early communication with employees was the potential rejection of the project at an early stage and the feeling that the project is not worth communicating in its early stages, since the outcome would not be clear. Moreover, the companies were unsure about the extent to which different stakeholders within their companies should be involved in general and did not see the added value of open communication at an early stage.

In addition to the company-internal communication, there was constant communication within the project teams, which include researchers as well. Communication determines the target-oriented information about the project status to the previously defined actors as well as the regular exchange between the project partners. In all companies, there was also a regular exchange with the interdisciplinary consortium, which ensures the exchange of expertise and the monitoring of the current project status. These meetings took place in either two-week or weekly cycles depending on the company. Overall, an iterative approach to communication has proved to be very beneficial. Continuous feedback loops made it possible to quickly react to changes in the project process. Regular updates made it possible to derive individual requirements for specific key topics and workshops (e.g., data management).

The results indicated the need for a process to involve employees during the project cycle. This includes the perspectives from different target groups, like workers, customers, team leaders or managers.

5.2 Stakeholder perspectives

In the work with the companies during the first three DSR steps of their AI projects, it is concluded that there are hardly any practical tools for companies to reflect on the impacts of their envisioned technical solution. In different workshops, awareness for a holistic view on AI was built and the human-centered approach was strengthened. A reflection on stakeholders, defined as “an entity with a stake (interest) in the subject activity” (McGrath and Whitty 2017) is useful for classifying who would use an AI system and who not on the one hand, as well as who would be directly affected by the system and who would be indirectly affected by it, on the other hand, is particularly interesting. To support this reflection process, a matrix (Fig. 4) for stakeholder mapping is suggested as a tool to differentiate the stakeholders and the degree to which they are affected by AI. This matrix is a useful tool for management, works councils or developers to assess the impact of the planned AI project.

Fig. 4 Abb. 4

Matrix for the classification of stakeholder groups (SG) for AI projects

Klassifikations-Matrix für Stakeholder-Gruppen (SG) in KI-Projekten

The three companies presented here were matched to the matrix each (Fig. 5):

Fig. 5 Abb. 5

Practical implementation of the stakeholder matrix for the three companies

Anwendung der Stakeholder-Matrix auf die drei Unternehmen

In Company 1 and 3, stakeholders who interact with the AI are directly affected as well, whereas Company 2 is the only one where the users and those affected are disjoint groups. Employees who are directly affected but do not interact with the AI itself are, in all three companies, employees who are on a lower hierarchy level. It seems very important to pay particular attention to communication with this group to eliminate fears and worries and to create options for participation. The suggested matrix is an adoption to a classical stakeholder analysis that is performed with the aim of identifying all those persons or groups that are impacted by a project or have an impact on it themselves.

Employees who were not affected by decisions of the planned AI model were included in the process only late or not at all, which was due to common considerations and, above all, the provision on the part of the company. It is worth considering if and when this group should be integrated into the process in the future. When immature tools are presented to non-users, there is also a fear of creating disruption and frustration among these employees. Hence, the processes predominantly involved team leaders and affected employees. However, when asked, employees typically pronounced inclination to participate due to the possibility of shaping the future of their own work.

Another finding concerns the scope of the AI projects in the respective companies. There are relatively small and limited projects and those which are widely extended. In companies 1 and 3, the AI system should support a delimited process at the beginning of the complete workflow. This contrasts with company 2, which wants to optimize a wide extended process with many features involved. This is neither a quality feature nor the amount of the expected reduction in workload. Mainly the envisaged goals, but also the degree of digitalization and the data influence the scope. It is considered a coincidence that the largest company shown here also has the largest project.

6 Discussion and future work

The presented paper examined the research question “How can human-centered design and development of technical innovations, especially AI technologies, be carried out in a practical and scientifically sound manner?” For this purpose, three practical use cases of pilot companies were considered, and their human-centered introduction of AI was analyzed. This included the participation of different target groups (who participated and to which extent) as well as the methods used during the DSR process.

It is primarily the consideration of employee feedback and implementation suggestions that is decisive for the motivation and acceptance of the system and not the quantitative extent of participation throughout the project (Ullrich et al. 2023). However, Fig. 6 illustrates that in practice not all employee resp. stakeholder groups, including those directly affected, were involved right from the start of the initial conception phase (DSR step 1 and 2) but different stakeholder groups were integrated into various design phases. The reason for this is that in practice, feasibility and effectiveness must be taken into account. Nevertheless, this should not be a justification for ignoring the employees affected. In Company 2, employees were also informed indirectly in step 1 and step 2, and suggestions were included, but this was done via the individual shift planners. In Company 3, the production employees, who are not expected to use the system, have been informed of the project via the company’s internal communication channels. Given the small sample size, these observations cannot be generalized. Therefore, it is suggested that the role of stakeholders in AI projects does not determine their inclusion within a specific design phase. Future research should focus more on stakeholder integration (engagement) and organizational communication/participation to better understand their impact and relation on technology acceptance¹ and the overall performance.

Fig. 6 Abb. 6

Stakeholder matrix and inclusion of stakeholder groups (SG) during the DSR process

Stakeholder Matrix und die Einbeziehung der Stakeholder-Gruppen (SG) im DSR-Prozess

However, in the first project phase, the focus is on determining the initial requirements in accordance with the DSR. With a human-centered approach, this should also consider agreements on employee integration and systematic inclusion of works councils. In the presented analysis, this initial assessment was carried out in all companies by conducting interviews. Another starting point for determining the current situation is maturity models. However, studies show that traditional maturity models are often not sufficient to adequately integrate sustainable and social aspects. With the paradigm shift to Industry 5.0, new maturity models will be developed in the future which, in addition to personal interviews, provide an easily accessible overview of the as-is status and practical recommendations for the companies (Hein-Pensel et al. 2023).

There is also no uniform recommendation regarding methods for employee participation, as this depends on the company structure. Some companies inform employees at an earlier project stage, which can lead to better acceptance in the following steps. At the same time, involvement at an early stage, when precise goals still need to be defined, could contribute to uncertainties. These effects and the resulting optimal degree and methods of involvement must be investigated further in future research. Regarding this, the analysis of the use cases indicated that an iterative approach with regular communication and feedback loops can be particularly helpful in recognizing individual needs in advance. Other approaches to human-centered AI implementation, for example by Shneiderman (2022), have already shown the crucial importance of an iterative approach. This was practically confirmed during the conducted study.

Through a regular exchange, individual requirements (e.g., for workshop concepts or additional meetings) could be derived in a needs-oriented manner and integrated into the process in an agile way. The methods used in the step “communication” are mainly assigned to the areas of continuous information and knowledge transfer. According to Hevner et al. (2004), this transfer must be both technology- and management-oriented. The methods currently used in all companies relate thematically to both technological implementation and social factors.

6.1 Practical guideline

The study showed that the combination of the DSR approach together with four identified key aspects of a human-centered approach can successfully support AI projects. The current project results show that the introduction must always be adapted to the individual objectives and circumstances of the company. However, it was found in all projects that a holistic view (i.e. the inclusion of all levels of employment, see Chap. 5.2), transparent communication (Felzmann et al. 2020) and the safeguarding of ethical aspects (High-Level Expert Group on Artificial Intelligence 2019) play a central role in achieving the objectives. Due to the iterative process, all aspects are considered in regular feedback loops. The key aspects (Fig. 7) are goal defining & target groups, holistic view, transparency and ethical aspects, they are described in more detail in the following.

Fig. 7 Abb. 7

Key aspects of the human-centered design and development of AI systems

Zentrale Aspekte der menschzentrierten Gestaltung und Entwicklung von KI-Systemen

To bring the human-centered DSR approach into practice, AI designers and developers, managers or works councils are encouraged to consider the following aspects and reflect on the according questions:

6.1.1 Define a goal and check whose goal this is!

In the beginning of the design process that begins with the problem description and objective definition, the following questions should be asked:

• What is the suspected outcome of the method/tool?

• Who needs the system?

• What result do the users need?

• Who is supported by the tool?

6.1.2 Keep a holistic view during the design and development process!

To achieve a solution that does not only meet the initial problem but also serves as a sustainable AI solution, the following questions should be asked and answered:

• Who are the users of the system?

• What do the users need?

• Are all the (directly and indirectly) affected people aware of the development process?

A tool that supports answering these questions is the suggested stakeholder matrix. By clearly positioning the different directly or indirectly affected stakeholder groups, it supports the communication between project teams and data protection supervisors or works councils about legal and ethical implications. Targeted measures, e.g. on data protection or transparency, can then be derived based on the localization.

6.1.3 Create transparency!

Communication is a crucial part of human-centered AI and should accompany the process from design to evaluation. As already discussed in the literature (Ullrich et al. 2023; Shneiderman 2022), involvement of employees can contribute to participation and support the successful transformation process that comes along with new technologies.

• What data sets are used for the training of the model?

• Is the AI system requiring ongoing data collection?

• Who provides the data?

• Who is (directly or indirectly) affected by the system?

• Do users understand the way in which the system works (understandability)?

• Where is the data processed and how is it protected?

• Does the collection of data affect the process/the employees?

6.1.4 Take ethical aspects into consideration!

When designing and developing AI systems, social and environmental aspects must be considered regarding the results or potential effects of automation as well as possible biases in the data sets used to train a system.

• What impact does the outcome have on the users?

• What are the suspected (positive and negative) outcomes on the individual employee/on working processes/on the company?

• Do the data sets include person-related data?

• Is there any bias in the dataset?

Figure 8 shows the assignment of the four identified key aspects of a human-centered approach to the first three DSR steps. Define a goal and check whose goal this is, are mainly important during problem identification & motivation and objective definition. During design & development, the consideration of transparency is essential to assure acceptance and contribute to requirements regarding explainability of AI, whereas the holistic view should encompass both definition and development. Ethical aspects should be considered in all steps.

Fig. 8 Abb. 8

Key aspects of the human-centered design aligned with DSR-Steps

Kernpunkte des menschzentrierten Designs entlang der DSR-Schritte

6.2 Limitations

Three companies were used as the basis for this practical guideline, which limits the adaptability of the results, particularly to companies with different objectives and company specifics. Nevertheless, the results mark a first step towards a generalizable approach to human-centric AI development, especially for SMEs. It should also be noted that steps four and five of the DSR are still pending, as the current project has not yet been completed. Although methods for demonstration and evaluation are already planned, no accurate statement regarding their applicability is possible in advance. As an important tool for checking the applicability, a comprehensive evaluation of the developed approach is also still pending. To ensure the transferability of the developed approach, it is essential to test it in other companies. Therefore, evaluations in companies with different organizational structures are necessary to validate its generalizability and practical applicability.

7 Conclusion

The aim of this research was to examine the practicalities of human-centered AI development, particularly in the context of SMEs and manufacturing industry. By an in-depth analysis of three distinct use cases with pilot partners, considerations are made for effectively integrating stakeholder involvement throughout the AI planning and development process.

Through an empirical approach grounded in the methodology (see Chap. 3), the paper shows human-centered aspects in AI implementation processes. Insights derived from this study reveal gaps in employee involvement which often occur late in processes. The practical implications of this study are particularly relevant in the era of Industry 5.0, which demands a more human-centric approach to technological development and digital transformation. The need for companies to conform to these human-centered paradigms is growing and beneficial to such development processes, necessitating a clear and actionable guideline for navigating the complexities of digital and AI-driven change.

The paper suggests an iterative, inclusive approach that begins with a clear goal definition and extends through continuous and transparent communication and feedback. The developed practical guideline, grounded in the DSR approach offers an operationalization of generally recognized research findings regarding Goal and Target Defining, Holistic View, Transparency, and Ethical Aspects for human-centered AI. The guiding questions are deducted from a multidisciplinary approach for the human-centered development and implementation of AI projects. It is important to note that the development process is highly individual depending on the company and should be tailored to their specific needs.

For future research, it is recommended that the findings be expanded through broader empirical investigations involving a diverse set of companies and organizational structures. This would confirm the general applicability and adaptability of the practical guideline. Furthermore, the dynamic and evolving nature of human-centered design mandates continuous refinement and re-evaluation of the methodologies and guidelines proposed in this paper. In essence, this paper analyses practical work in collaboration with SMEs, among others, and offers a starting point for SMEs to approach AI development more thoughtfully, prioritizing human elements for better outcomes in the digital age.