Keywords

1 Introduction

Artificial Intelligence (AI) has become increasingly pervasive in various aspects of human life, ranging from healthcare and finance to education and entertainment. The integration and successful implementation of AI systems, however, present considerable challenges. In Germany and Europe, the adoption of AI has been hampered by delays and obstacles, with notable implications for small and medium-sized enterprises (SMEs) [1, 39, 16]. These SMEs play a crucial role as drivers of the German economy but face distinctive disadvantages in AI implementation. To ensure that AI systems are effectively designed to meet the needs of end-users, a promising approach known as Human Centered Design (HCD) has emerged. HCD focuses on placing human considerations at the forefront of AI development, thereby improving the overall user experience and system performance. By embracing a human-centered approach, potential gaps, and areas for optimization in the implementation process of AI can be identified. This paper aims to delve into the realm of AI implementation, with a particular focus on the application of HCD principles. Drawing upon existing literature and relevant case studies, the paper seeks to illustrate the benefits of adopting HCD in AI development while highlighting weaknesses and optimization potentials within existing models. By analyzing these gaps, it becomes possible to propose recommendations for future research in order to enhance the effectiveness and impact of AI implementation processes. The subsequent sections of this paper will provide an in-depth exploration of the challenges faced in implementing AI systems in Germany and Europe, with a specific emphasis on SMEs. Additionally, the principles and practices of HCD will be examined to shed light on its value in addressing these challenges. The paper will also present case studies to illustrate successful applications of HCD in AI development. Finally, based on the insights gained from the analysis, recommendations will be put forth for future research directions in this area. By critically examining the implementation of AI and emphasizing the significance of HCD, this paper aims to contribute to the advancement of AI technologies that are efficient and aligned with the needs of end-users. Ultimately, this will foster the responsible and effective deployment of AI systems, leading to widespread benefits and advancements across various domains.

2 Methodology

To achieve the objectives of this paper, a systematic literature analysis was undertaken in March of 2023. A forward meta-analysis of papers was carried out, resulting in 500+ entries. The search was conducted using specific keywords related to HCD and AI implementation, namely “HCD+AI implementation.” By employing this approach, a comprehensive collection of relevant scholarly articles, conference papers, and other scientific publications was gathered. The search results were then reviewed and filtered based on predefined research criteria. A specific backward search was carried including the terminology “human focus” and “SME” which narrowed down the available literature sources and opened up the research gap. The criteria furthermore ensured that the selected sources were directly related to the intersection of HCD and AI implementation with specific focus on the human and the SME sector, providing insights into the challenges and optimization potentials in this domain. In terms of ensuring quality and topicality (in-depth papers not older than 5 years) of the research, peer reviewed (quality assessment) and highly ranked (topicality) journals were used for the in-depth analysis. The intention of this systematic literature analysis was to gather a representative sample of the existing body of knowledge on the topic. An effort was made to cover a wide range of publications to capture the diversity of perspectives and findings in this field. Through the analysis of the selected studies, common weaknesses and gaps in existing models were identified. These overlaps in weaknesses provided evidence that the identified challenges were not isolated incidents but rather prevalent across different AI implementation models. This leads to the conclusion that the weaknesses identified in the examined studies are likely to exist in other models as well. Based on the findings from the literature analysis, recommendations were derived to address the identified gaps and optimize the AI implementation process. These recommendations serve as valuable insights for future research and development efforts in the field of HCD and AI implementation, aiming to enhance the design, deployment, and user experience of AI systems. It is important to note that this methodology of systematic literature analysis provided a rigorous and structured approach to gather and analyze existing scientific knowledge. The inclusion of multiple scientific databases and the use of specific search terms ensured a comprehensive search process, while the predefined research criteria guided the selection and examination of relevant sources.

3 State of the Art

HCD is a process-oriented approach, according to Norman [2013], which focuses on designing products, services, and systems tailored to the needs and requirements of users. The HCD process involves several interconnected steps that may include iterative loops:

  1. 1.

    Understanding users: Comprehending the needs, desires and behaviors of users.

  2. 2.

    Defining problems: Specification of the requirements and constraints that must be considered during development.

  3. 3.

    Ideation: Generating ideas to solve the defined problem.

  4. 4.

    Prototyping: Implementing ideas into tangible solutions.

  5. 5.

    Testing and Feedback: Testing prototypes with users to obtain reactions and feedback.

  6. 6.

    Refinement: Re-adjusting and adapting concepts based on results from the testing step [cf. 36].

The integration of Human-Centered Design into the developing process of AI is the result of an increasing focus on designing technology that aligns with user needs and preferences. The integration of HCD-approaches into the AI-implementation process enables faster and more efficient data processing, as well as generating insights that can improve the user experience and support decision-making [cf. 53].

3.1 Comparison of Existing Models

Overall, there are various ways in which HCD and AI can intersect. The choice of approach depends on specific requirements, including goals, target audience, and available resources. However, careful integration of AI into the design process can help create better products and services that align with the needs and perspectives of users. HCD is a crucial approach to ensure that the development and implementation of AI systems are aligned with the needs and values of humans. However, there is no one-size-fits-all approach for implementing HCD in AI, and various models have been proposed by researchers and practitioners. This topic aims to compare existing models for HCD in AI-implementation, explore their strengths and weaknesses, and identify potential areas for improvement. By analyzing and synthesizing these models, optimization potentials can be conducted, which can be considered in future research (Table 1).

Table 1. Comparison of different HCD-AI-Approaches
Full size table

This paper mainly focuses on the approach of human-centered AI design. The further theories shown are seen as complementary to the primary approach. As mentioned above, implementation strategies of AI projects often lack a human-centered approach and neglect the consideration of humans during the implementation process [cf. 43; 21]. However, there are SME criteria that should be considered when implementing AI projects, such as limited resources, limited expertise, and often a lack of experience in implementing AI projects [cf. 49]. Although there are good models HCD, they are often not implemented, leading to inadequate implementation of AI projects. Overall, the models are a useful contribution to the theoretical implementation of AI, but the implementation of real use cases is problematic.

4 Soft and Missing Spots of Human-Centered AI Implementation

The introduction of AI in SMEs by using human-centered methods requires not only the involvement of humans in all considerations of technology introduction. Based on the indications of previous study, a socio-technical specification sheet proves to be effective as a starting point, in which the three dimensions of human-technology-organization are considered in the specific implementation project, here AI introduction [cf. 21; 40; 44; 50], as a human-centered design of the AI process cannot only focus on the consideration of human-technology interaction. The design of work processes by managers and organizational specifications must also be considered if AI is to be developed and used in a human-centered way [cf. 23].

4.1 Technology-Organization-Spots

The implementation of AI solutions in companies can bring new challenges. Weber et al. [cf. 51] assume that organizations are better positioned in this process if they have specific resources. These so-called organizational capabilities include AI project planning and development, data management, and AI model lifecycle management. The development of these abilities in the face of new and systemic challenges leads to unforeseeable processes in addition to the consideration of human-centered design in AI implementation. A study on user experience and usability among scientific developers showed that their orientation in the development of corresponding systems is not so much guided by user experience as by management and organizational issues [cf. 13]. For example, the successful implementation of AI in the education sector is recommended to be accompanied by the cooperation of researchers with local teachers so that organizational factors can be incorporated into the implementation process, which can affect the sustainable usability of AI [cf. 6]. Al Ali and Badi conclude that AI introductions should be accompanied by organization-wide change management processes because they can affect all employees and their work organization [cf. 2]. The question here is how technological and organizational processes can be integrated to design the introduction of AI as a supporting and – in some cases – disruptive technology. This presents SMEs in particular with additional challenges. SMEs should plan to implement the understanding of the AI implementation process as well as ensuring human-centeredness in day-to-day business [cf. 37]. Therefore, the approach of using an HCD analysis when implementing AI [cf. 53] is interesting. However, the effects of AI use are felt across departments and must be incorporated into business processes. Therefore, it is worth considering whether focusing on HCD regarding the directly affected users is sufficient or whether the involvement of proprietary stakeholders should be considered (e.g., managers, HR developers, [cf. 3]). Although the national AI strategy for Germany recommends responsible development and use of AI, the use of AI in Germany lags forecasts [cf. 18]. The SMEs predominant in Germany view the use of new technologies such as AI skeptically and rely on conventional technologies, such as rule-based systems [cf. 45]. A human-centered introduction of AI in German SMEs must also consider organizational restraints and the usage behavior of previous technologies to ensure a human-centered AI development process. However, it is also to be considered whether human-centered factors of the AI implementation process, as well as ethical considerations, could be implemented as dangers in the life cycle of an AI system [cf. 50].

4.2 Human-Technology-Spots

As stated, it is recommended to design the process of implementing AI with a human-centric approach. This also involves the human-centered design of human-machine interfaces as the connection between humans and technology in the company. For example, the ability to use AI is also a design issue, which can be ensured by early and transparent involvement of users in the development and design process of AI itself [cf. 31]. On the other hand, in the study of “Participatory Design in the Engineering Design Educational Environment,” user involvement is not participatory in development, but tested through experiments afterward [cf. 12]. Technology acceptance also depends on whether the personal needs of users are considered, which includes process knowledge about the possibilities of AI and participation in defining the problem background [cf. 52; 38]. For example, it is part of the process to determine the scope of involvement of a worker together with the work planner and production manager for a human-centered introduction of AI [cf. 38]. However, this approach in practice faces limitations when workers cannot be released from the company’s perspective (e.g., staff shortages) or when conflicts arise regarding the interpretation and legitimation of human-friendly work design [cf. 9]. The implementation of ethical and human-centered design guidelines must be evaluated and negotiated in corporate practice. Corresponding guidelines are defined as principles that must be translated into the daily work of employees, managers, and executives [cf. 37; 47]. Currently, there is still discussion about which competencies are even useful for human-centered AI use, which roles AI and humans take on, and how complex computational processes should be taught from a learning theory perspective [cf. 17; 35]. Regarding proprietary users (in the work system environment) in the company, such as executives and personnel managers, this knowledge is relevant to support workplace-related competency development processes. They can only become role models for digital transformation in the company if corresponding gaps are closed.

4.3 Human-Organization-Spots

However, the need for competency development among members of the organization extends not only to AI application knowledge regarding the technology solution to be introduced. Rather, the requirements for interdisciplinary skills associated with changes in work processes and responsibility structures are also increasing. A new leadership-, error-, and knowledge culture is considered essential in the AI implementation process to make the disruptive technology beneficially used in the company [cf. 21]. Regarding the roles negotiated in the socio-technical system between humans and the organization, the following requirements arise, which are inadequately considered in existing AI implementation models. Competency development is often implemented as AI training, but informal learning and creative processes in the team and workplace, and thus the integration of the company’s learning process into the AI design process, are particularly important for adaptation and fit [cf. 32]. Employees and managers are equally affected by the transformation of the digital working world. For managers, it may become necessary to establish a new leadership culture [cf. 19]. Herrmann and Pfeiffer speak of organizational embedding of the AI implementation process when changed role requirements as well as rules for dealing with AI results and constraints must be considered in the everyday work of managers and HRM [cf. 2022]. In the area of AI implementation, there is increasing discussion about whether AI is attributed a social role. The use of ML algorithms as affective computer agents that interact with work groups through suggestions and participation in conversations, or that can proactively detect emotional and stress situations, is a growing topic within the team awareness discussion [cf. 48; 41]. The integration itself as well as the recommended methodology within the process of integrating employees into the competency development process in the context of AI applications is still unclear. The issues range from accounting aspects and the question of cost-effective integration of employees to the determination of performance indicators for tracking learning progress. Competence development as such remains a permanent topic in organizations, which is further reinforced by personnel shortages and lateral entrants. In addition, there is a backlog in the development of digital competences, especially in SMEs. In SMEs, qualification progress is tracked more manually and therefore not updated daily, and soft skills are only partially mapped [cf. 42; 5]. The measurement of knowledge and understanding of AI applications, especially in the competency development process of employees, is unclear [cf. 41]. Currently it cannot be adequately proven at what point an employee has sufficiently understood the AI to be able to apply, operate, and monitor it. The operationalization of experiential knowledge is not finally clarified [cf. 41]. Studies on digital skills in German SMEs also show that digital competencies are not being implemented mainly due to lack of time and high costs [cf. 34]. An evaluation by the OECD of 100 case studies on AI implementation, however, showed higher competency requirements that accompany the use of AI [cf. 33]. In addition, it becomes clear that the role of informal learning in the AI implementation process is often underrepresented, although it offers potential for human-centered adaptation of AI in the workplace and in the team. A shared definition of the role of AI in the organization and a clear operationalization of the competency development process are necessary to successfully implement AI [cf. 32]. In conclusion, the main challenges of implementing AI lie in digital skills, but also in the acceptance of organizational change. According to the HTO (human-technology-organization)-system [cf. 44] from a work design perspective, the role of humans remains unclear. This means that the relationship between humans and the organization is not co-developed – Human-Machine-Interface (human-technology) is considered as well as socio-technical systems (technology-organization), but not H-O (human-organization). This is especially problematic due to the overload of H-T (human-technology) design research before H-O design research, which risks prioritizing technological design over human-centered design, contradicting the principles of HCD.

5 Optimization Potentials and Solution Approaches

Building on the soft and missing spots in current models, this paper presents solutions that are currently being tested in German business practice within a practical AI project.

The Competence Center for the Future of Work in the “PerspektiveArbeit Lausitz” (PAL) project investigates how AI solutions can be implemented in a practice-oriented manner. The aim is to introduce data-based assistance systems, including artificial intelligence, in small and medium-sized enterprises in Lusatia in Saxony and Brandenburg in order to cope with structural change. This is a practical project because the AI solutions are ensured through participatory involvement of teams from four universities, 23 companies, and associations. The “PerspektiveArbeit Lausitz” project is being funded within the framework of the funding guideline “Future of Work: Regional Competence Centers for Labor Research. First round of competition: Design of new forms of work through artificial intelligence” in the program “Innovations for the production, service and work of tomorrow” by the Federal Ministry of Education and Research under the funding number 02L19C306 with a project duration of 01.11.2021 – 31.10.2026. The Lusatia region is characterized by three major challenges, the so-called “3D”: demographic change, decarbonization and digitalization. Demographic change is reflected in Lusatia by a sharp decline in population and an aging population. The population of Lusatia fell by around 25% between 1990 and 2019, while the proportion of people over 65 rose from 15.5% to 26.1% [cf. 55]. This development also has an impact on the labor market, as there is a threat of labor shortages. Decarbonization is another challenge for Lusatia, as the region has traditionally been characterized by lignite mining and electricity generation. However, the energy transition requires a move away from coal and the expansion of renewable energies. This poses major challenges for the economy in the region and requires investments in new technologies and infrastructures [cf. 55]. Digitalization is another challenge for Lusatia, as the region has so far been less developed in this area than other regions in Germany. SMEs in the region, in particular, are struggling to meet the demands of digital transformation and need to catch up in this area [cf. 54]. These challenges require a targeted strategy and investments in the region to maintain competitiveness and create sustainable jobs. Based on the challenges presented by the “3D” in the Lusatia region, action needs to be taken to address these issues and ensure the region’s competitiveness and sustainability. Deriving from the soft and missing spots in the AI implementation models and the experiences from the project PAL, four recommendations for action can be derived as to how successful AI implementation can be accompanied.

5.1 Overall: Reflection Loop

To ensure effective reflection and learning, it is important to not leave the HTO reflection loop until the end of a process, but instead reverse the process and incorporate reflection as an initial step [cf. 8]. As shown in the PAL-CRISP-DM model, an optimization approach of the CRISP-DM-model, it is possible to incorporate reflection loops throughout the whole process which meet the first optimization potential (see Fig. 1).

Fig. 1
figure 1

AI procedure model [20, 10]

Full size image

The introduction of data-based assistance systems or AI applications in companies often faces the challenge that employees are skeptical or even critical of such projects. In addition to the fear of losing their jobs due to technology, in many cases, technophobia plays a role, especially among older employees who are afraid of not being able to meet the resulting requirements. It is therefore imperative to involve the employees directly affected by the project in the conception and implementation of the project from the beginning and not only to “take them along” in the sense of “informing” them, but to involve them in the implementation of the project and to include their know-how. A first prelude for this can be an AI acceptance workshop in which, on the one hand, the legal framework conditions (for example, relevant aspects of the GDPR) and, on the other hand, the management’s expectations of the project as well as a description of the project are discussed with the employees involved. Ideally, this workshop is designed to be low-threshold, for example, by first discussing the positive aspects of AI applications and examples of using AI solutions in everyday life.

It is also helpful if the company agrees to a code of conduct for the application of AI in this context. For example, the “Human Friendly Automation Value Manifesto” is a suitable option. Such a code of conduct can visibly communicate the company’s approach to the application of AI both internally and externally, thereby providing employees and potential applicants with a trustworthy and secure interaction with the new technologies. Such an AI workshop should then also be conducted with other employees in the next step. This way, uncertainties and resulting unrest in the workforce, which becomes aware of the project but has no detailed information about it, can be avoided early on. The main message of such a workshop, to which the company management must also position itself clearly, should be: “We are shaping this process together!”

5.2 Human-Organization: Competence Development Needs Misjudged

To meet the challenges posed by the changing demands of the digital age, it is necessary to develop new qualifications and competence development models. Moving away from individual training to the integration of multimodal learning approaches, as well as process support, is crucial for employees to remain willing to engage in lifelong learning. According to a study by the European Commission, learning in the workplace is crucial for employees to adapt to changing requirements and maintain their employability [cf. 15]. To achieve success, non-formal and informal learning must be planned and structured directly with the help of team processes and reflection loops, with iterative reflection loops being a key factor. Research shows that reflection loops can be an effective tool for improving learning outcomes and enhancing employee performance [cf. 29]. This involves considering not only technological feasibility but also the design of informal learning opportunities. Process thinking is essential for employees to move towards lifelong learning. However, digitalization in public administration can be hindered by factors such as process chains from the 1970s, which cannot be solved through training. Therefore, process support and project management are required to help employees overcome such obstacles in both the market and the workplace [cf. 24]. Job rotation is a potential solution to help employees adapt to changing demands. Studies have shown that job rotation can have a positive impact on employees’ skill development, job satisfaction, and career advancement [cf. 46]. Furthermore, the example of introducing AI-assisted systems highlights the importance of planning the timing of competency development. Although these systems can demonstrate their functionality and benefits through use and application, the understanding of the processes must be established beforehand. Otherwise, an informed decision to participate in the adaptation of the AI solution cannot be presented. This means that necessary competence development steps must already be carried out in the company before the test phase, so that the employees are empowered sufficiently. Research suggests that involving employees in the development and implementation of AI solutions can increase their engagement and commitment to the technology [cf. 26]. Within the framework of the project PAL, a participatory approach is adopted in project and process management, with an initial workshop involving employees, managers, and scientists to address any initial questions, create acceptance, and establish initial process understanding, especially for SME [cf. 21]. Including the (learning) experience of users in the implementation process addresses all optimization potentials of AI approaches.

5.3 Human-Organization: Termination Criteria and Inclusion

In the implementation of AI systems, it is of great importance to define termination criteria in order to minimize possible errors and risks. Recommendations for discontinuation criteria can be based on, for example, error rates, trustworthiness of the data and the application, ethical considerations, and regulatory requirements. It is important to strike a balance between avoiding errors and the benefits of the application. Too restrictive an approach can limit the benefits of AI systems, while too open an approach increases the risk of errors and harm. Terminations criteria must be defined at the beginning of each process. For instance, agile lab teams can be defined, that accompany the process on a permanent basis and are responsible for checking the termination and success criteria regularly [cf. 19]. This supports transparency and collaboration between developers and users, as called for in the second and third optimization potential. The inclusion of employees in the process should also be considered in order to achieve a high level of acceptance and participation. This can be done through training, workshops, and participatory approaches. A high level of employee participation can help identify and address concerns and challenges early on. Active involvement can also promote employee acceptance and motivation, which in turn supports the success of the implementation. It is important to note, however, that employee inclusion should not be seen only as a means to an end. The inclusion of employees in the process should be pursued as a goal in its own right, as this can contribute to a more equitable and inclusive working environment [cf. 11].

5.4 Technology-Organization: Further Development of Success Indicators

In addition to Key Performance Indicators (KPIs) that measure the number of loops and areas of congestion, it is also important to use satisfaction scores as a measure of the success of AI implementation in a company. Customer and employee satisfaction are crucial in ensuring the effectiveness of the AI system and can provide valuable insights for further improvement [cf. 22]. AI can help improve customer experience and satisfaction by providing personalized recommendations, faster response times, and more efficient service. This can be measured through customer feedback surveys and ratings. Furthermore, AI can help improve employee satisfaction by reducing mundane tasks, increasing autonomy, and providing better decision-making support. This can be measured through employee feedback surveys and retention rates [cf. 14]. Companies should aim to empower themselves to make further adjustments to the system without relying on external support. This involves a deep understanding of the implemented AI technology and developing internal expertise for system maintenance and improvement. As part of the company-specific transformation plan (see Fig. 1), satisfaction metrics support the evaluation of the project, which addresses the fourth optimization potential

6 Conclusion and Outlook

Considering the theoretical considerations, the view on the implementation figures of AI in German SMEs, and the practical application in the PAL project, a human-centered design for AI implementation can be complemented as following: Human-centered design considers all persons affected by the product, including executives and personnel developers beyond the boundaries of the work system [cf. 9]. Furthermore, the connection between HCD and HTO is beneficial for practice, as it reveals gaps and solutions in the development of Human-Technology design. However, the practical relevance of HCD has not been adequately evaluated. For instance, what constitutes sufficient involvement of affected parties? Is feedback from users sufficient, or should proprietarily affected parties, such as executives and leaders of other departments, be included? Therefore, a participatory approach should be adopted from the outset to incorporate practical use cases. Particularly, the neglect of reservations and technology experience of executives and management should be considered in AI implementation processes. A human-centered AI implementation is always an intervention in the prevailing work system and therefore organizational processes. Especially in the Competence Centers for Labor Research in Germany, which focus on the design of the use of methods and tools of artificial intelligence, there is a chance of human-centered design of future work. At the same time, the requirement for a human-centered development and introduction of AI requires caution with the obstacles highlighted. Especially for SMEs, this offers the opportunity to try out the use of AI and reduce the digital divide through AI [cf. 27]. This opportunity is particularly important in view of the unclear risk assessments of AI use, in order to open up usage perspectives for SMEs [cf. 30]. The strength of the contribution is the consistent HCD along established socio-technical dimensions. The solution approaches of AI-Implementation should be further developed [cf. 8].