1 Introduction

Humans’ fear of being replaced by automation dates back to the ancient world, where Aristotle questioned the necessity of slavery if humans were able to develop technologies that can replace their labor (Chase 1958). Ever since, this fear has returned constantly. Aristotle’s thoughts have been revitalized around the time of the Industrial Revolution, when the development of new machines triggered the fear of job elimination and resulted in protests and destruction of these machines by the workers. However, Keynes emphasized the advantages of the development of new technologies, which save time on occupational tasks and thus increase leisure (Keynes 2010). Furthermore, modern technologies also create new tasks in other fields, which keeps the rate of employment from falling (Acemoglu and Restrepo 2018).

Nowadays, modern technologies that trigger the fear of job replacement are automated algorithms that respond to customer inquiries, self-driving cars and robotics that increase economic efficiency (Brynjolfsson and McAfee 2011; Frey and Osborne 2017). Forty-nine percent of global jobs have the potential to be replaced by advanced technologies (McKinsey 2017). This is particularly true for jobs with well-defined and repetitive tasks, as their procedure can be taken over by machinery or codified in computers (Acemoglu and Autor 2011). Meanwhile, jobs requiring academic degrees are expected to be of high importance for the global economy due to the non-replaceable value of the knowledge they require (McKinsey 2017). In addition to future demands for this kind of jobs, social and communication skills are gaining significance. This can be explained by the flexibilization of the organization of jobs as a result of the opportunity to work independently of geographical location and time (Ramioul 2007). Due to this decentralization of tasks, the success of a project strongly depends on an efficient communication system for workers who work in different countries on the same project. These emerging demands require the development of new skills.

The Organization for Economic Co-operation and Development (OECD) conducted research on skill demand and supply with regard to changing labor markets and came to the conclusion that approximately 19% of workers have insufficient skills for their job. Due to this lack of skills, it is crucial to provide workers with additional competences to prepare them for changing job requirements and for possible job losses due to digitalization.

Efficient training and real-time on-the-job support require user-oriented technologies to enhance the human intellect (UK Commission for Employment and Accounts 2016). This phenomenon is known as intelligence augmentation (IA), which can be understood as an advancement of human abilities based on technological devices that link digital data, graphics and services to physical movement and interactions in the real environment. An example of IA is augmented reality (AR), which provides an interactive and real-time user interface to an electronically enhanced physical world (Azuma et al. 2001). Augmented reality devices such as the Microsoft HoloLens overlay three-dimensional information and data on the user’s physical space and simulate interactions via a camera and corresponding software (Funk et al. 2017). Thus, AR can improve workers’ efficiency by providing real-time guidance. Another form of IA is virtual reality (VR), which refers to a computer-generated virtual environment that enables users to immerse themselves in any situation (Jayaram and Ibrahim 1999; Sacks et al. 2013) and it can train individuals with insufficient skills.

Some research has been conducted on the labor market risks due to new digital industrial technologies and artificial intelligence (Degryse 2016; Kessler and Buck 2016; Makridakis 2017). However, it is necessary to analyze not only how many jobs will be eliminated by technologies like AI, but also to provide and support concepts like IA with which we can soften the radical changes that might come in the future.

Thus, we investigate how AR and VR can help individuals’ transformation to a new occupation or provide them with a new skill set that is required in the emerging employment landscape in Germany. Even though the transformation of jobs due to technological developments is a global matter, we limit our analysis to the German labor market.

The remainder is as follows. In Sect. 2, we define IA and introduce the technological disciplines behind AR and VR. We also provide a brief overview of the current situation of the German labor market situation and analyze the job activities of identified job clusters regarding their potential to be automated to help identify suitable areas and processes in which innovative technologies, such as AR, can support employees in daily working activities. We conduct the systematic literature review as well as a practically oriented study on augmented and virtual reality use cases in Sect. 3. Based on these analyses, we establish a theoretical framework in Sect. 4 to guide an appropriate application of AR and VR in the workforce context. Finally, we summarize our findings, draw a conclusion and provide an outlook for further research.

2 Theoretical background

2.1 Definition and delimitation of intelligence augmentation

Dough Engelbart first coined the term “Intelligence Augmentation” in 1962. In his work, he examined how to augment the human intellect by “increasing the capability of a man to approach a complex problem situation” (Engelbart 1962, p. 1). To do so, they can use different methods to augment their capabilities. For example, humans can train themselves to use “artifacts,” “language,” or different “methodologies” in problem-solving strategies (Engelbart 1962, pp. 20–21). Engelbart’s goal was not to amplify the native human intelligence but rather to augment the problem-solving process with all its components.

We focus on leveraging human capabilities by using artifacts. These can be understood as hardware either to overcome physical obstacles or to provide support through technological software-based augmentation (Xia and Maes 2013). We concentrate on the latter interpretation of the term, as this kind of augmentation provides humans with the ability to solve problems that require high cognitive activity, which can be too complex without any support (Engelbart 1962). Since then, computer technology has made remarkable progress generating various opportunities for the enhancement of human intellect. Engelbart mainly focused on promoting human intelligence by amplifying its problem-solving ability. However, individuals also benefit from augmentation in other cognitive domains such as memories, motivation, or decision-making processes (Xia and Maes 2013). A central issue of this work is the vocational training and retraining of employees. To learn new skills, workers have to memorize new processes. Therefore, the augmentation of human memory in the form of a “conscious recollection about facts and events” (Squire 1992, p. 232) is particularly relevant in our work.

As pointed out, AR is well suited for the augmentation of human capabilities in the work context. Several experimental results indicate that visualizations of information improve the learning process of individuals, especially when combined with immediate interaction and practice of the theory (Rieber 1990; Mayer and Moreno 2002). AR display technologies can illustrate different components of a task, demonstrate transitional processes and allow the user to interact with them (McClean et al. 2005), meaning that AR can advance the skill development by consolidating the attention and memory of employees. Another advantage of using AR is the real-time access of information at the respective workplace which allows users to not retain every process step or all the data needed for the job. Instead, they can easily retrieve relevant data while completing a task. This real-time access to data allows a reduction of cognitive memory functions and the demands of completing the task. Thus, AR is suitable for enhancing employees on-the-job (Kohn and Harborth 2018).

VR, as the second form of IA investigated in this work, can be categorized into immersive and non-immersive systems. The latter provides the user with a virtual experience that is typically presented on a computer screen while the user remains cognizant of being located in a physical surrounding (Robertson et al. 1993). In contrast to non-immersive VR, immersive systems involve a computer-generated three-dimensional landscape in which the user is completely bound to the virtual environment (Azuma 1997). A distinctive feature of such a system is a VR headset with which the user can move her head, look around and see the rest of the simulated word similar to the real world. Various studies have shown that this VR experience enhances human learning processes with experience (McLellan 1996; Salzman et al. 1999).

2.2 German labor market

The unemployment rate in Germany has continually fallen since the economic crisis of 2008/2009 and has currently reached its lowest level in almost 30 years (Fred 2020). Since the crisis of 2008/2009, it has reacted relatively weakly to macroeconomic ups and downs, whereas other countries in the Europe have shown stronger employment fluctuations (Rhein 2010; Klinger and Weber 2014). According to the Institute for Employment Research (IAB), the low unemployment rate can be linked to the shortage of skilled workers, which encourages firms to recruit personnel irrespective to the economic situation (Bossler et al. 2017).

However, the impact of the structural demographic development has gained increasing significance for the German labor market. According to the Federal Statistical Office (2020), 7.6% of the labor force were aged between 65 and 70 years in 2008; this increased to 17% until 2018. This negative demographic trend can offset by high immigration; however, as immigrants often have a different level of education and qualification, it takes time until they can take on a job (Buslei et al. 2018).

Moreover, the German labor market is confronted with the effects of digitalization. A recent study on potential workforce developments until 2030 compares a “basic scenario” of a likely development of the labor demand and supply with an “Industry 4.0 scenario” incorporating the effects of the digitalization (Wolter et al. 2016). The results show that the onset of the Industry 4.0 scenario would lead to a loss of 1.46 million jobs in 2035 compared to the basic scenario. However, this scenario would also create approximately 1.4 million additional jobs in other fields (cf. Appendix Fig. 2). Although almost no effect on the number of employees can be observed, prior work concludes that dynamics in the overall employment would increase and that requirement profiles for workers could change (Warning and Weber 2017). Thus, we can expect a shift from low-income jobs requiring lower levels of education toward to jobs requiring higher levels of education. This shift has an unforeseeable social explosive force. Therefore, we analyze the concrete occupational cluster in Germany in which anticipated changes may occur in order to provide a basis for our consequent technological assessment on chances regarding the viability of IA.

2.3 Occupational cluster and their automation potential

The Federal Employment Office (BA) lists around 4,000 different types of occupations (Berufenet 2020a). As an analysis of the substitutability potential of each of these occupations would go beyond the scope of this work, we cluster these professions based on the occupational segments classified by the BA. The first classification level of the report consists of 10 occupational areas. As this categorization is too broad, they are not sufficient to reflect the occupational differentiation of the labor market in Germany. In contrast, the 37 main occupational groups of the classification report are too detailed for our analyses (Matthes et al. 2015). As previous research has shown, not entire professions but only activities can be replaced by computer-controlled machines (Bonin et al. 2015). Therefore, we use the 14 segments formed by homogeneity in tasks and activity, irrespective of school, vocational training, or position in the occupation or company. The clusters Manufacturing and Production Technologies have very similar and overlapping tasks, such as the maintenance and installation of machinery. Thus, we merge these clusters for the consequent analysis.

To identify potential occupational shifts, we analyze these activities concerning their potential to be automated by computers. We understand automatable activities as “task[s] that can be fully specified as a series of instructions to be executed by a machine” (Acemoglu and Autor 2011, p. 1076). The measure was developed by IBA scientists for the years 2013 and 2016 (Dengler and Matthes 2015, 2018). They calculate the proportion of core activities in a profession for which a mechanical alternative exists. The measures can give us an indication of which activities might be replaced by machines. Consequently, we can identify where the “upskilling” of the workforce is required to keep up with emerging needs in task capabilities through the rapid development of new technologies. We illustrate the clusters in Table 1.

Table 1 Overview of occupational cluster with respect to typical job activities and their potential for substitutability (Matthes et al. 2015; Berufenet 2020a)
Full size table

The highest potential for substitutability in both years occurs is in the Manufacturing and Production professions, which include mainly repetitive tasks, such as installing and maintaining machines. Furthermore, a large change occurs to Transport and Logistic professions, which recorded an increase in automatable job activities by 20%. In particular, tasks regarding the material flow within a supply chain system, such as transporting, handling, or delivering products, can be taken over by automatic guided vehicles (Karabegović et al. 2015). Moreover, the proportion of activities which can be substituted in Business-related Service professions experienced a significant increase since these tasks can be easily automated by computers.

In addition, 57% of the tasks of Managers and Executives could be substituted in 2016. This can be attributed to the fact that currently a large part of these tasks remains administrative. However, there are predictions that these professions will continue to exist in the future but will be subject to profound changes in tasks (McKinsey 2017). They expect that, in the future, managers will spend less time on reporting and monitoring tasks and will rather concentrate on leadership tasks that require social skills, such as stakeholder management or the supervision of employees.

In contrast, IT and Scientific Services professions show a decrease in job activities that can potentially be substituted. Designing and engineering tasks require high expertise in the specific application area and creative problem-solving skills, which cannot be easily replaced by machinery. Furthermore, the exponential growth of available data demands its analysis and evaluation to utilize them for business reasons. Thus, new jobs such as Data Scientists, which require high cognitive abilities, are demanded (Berufenet 2020b).

In Medical and Non-medical Health professions, some occupational activities can be substituted by new technologies. For example, 3D printing is used in the prototyping process of a patient-specific prosthesis (Cadd et al. 2018). However, other occupations assigned to this segment experienced a decrease in the proportion of activities that can be automated. In particular nurses and other caring professions require high social abilities, which machines cannot take over. The same holds for occupational segments which include education and entertainment of people show a relatively low proportion of tasks that can be automated.

Overall, evidence suggests that jobs which require high expertise and cognitive availability seem to be less susceptible to automation. Jobs that involve human interaction or creativity skills, for example, in education and entertainment activities, will be among the activities in which humans will continue to be superior to technical alternatives (Frey and Osborne 2017). Therefore, they are considered as low in automation (below 30%). By contrast, administrative and analytical jobs with high repetitive activities are particularly susceptible to automation. Furthermore, jobs with predictable physical operations, such as cleaning or assembly tasks that do not involve any customer interaction, or jobs which involve activities based on quantifiable data are subject to a medium to high automation risk (over 30%). These jobs usually require only low education, whereas jobs with the least substitutable activities require tertiary education (Nedelkoska and Quintini 2018).

In sum, significant changes for the future labor market are expected, resulting in employment losses in some occupations and in changing activities within a profession for other occupations. Irrespective of the actual extent of the employment losses, the main consensus is that the segments with well-defined procedural tasks in particular are more likely to be automated. The remaining and newly emerging occupations require higher qualifications and thus increase the pressure on the individual workers to adopt new skills.

3 Systematic literature review

3.1 Theoretical research

We conducted a systematic literature review (vom Brocke et al. 2009) to gain a wider understanding of possible solutions to enhance the human workforce. As a first step, we used two databases: ACM and IEEE Digital Library. As this work concentrates on AR as a form of IA, we decided to use Augmented Reality instead of Intelligence Augmentation as a keyword for our theoretical search. To specify the search, we added Workforce as a second keyword. Other keywords such as Education are not appropriate for our research question, as many results in collaborative learning methods in schools or universities are generated (Hughes 2005; Jantjies et al. 2018) but are not relevant to the enhancement of human capabilities at their workplace. To find information on VR, we conducted a second search with the keywords Virtual Reality and Skill Training. As we found several relevant papers which were not covered by our search in these two databases, we decided to generalize our search by using Google Scholar as the broadest database available to double-check our findings and add other potentially relevant papers. In addition, we broadened the keywords by using mixed and extended reality as they are a kind of umbrella term for AR and VR and combined these two keywords with vocational training to find work-related research articles (Mixed Reality and Vocational Training, Extended Reality and Vocational Training).

The final step includes a backward search for all relevant findings of the keyword search. First, initial hits based on the predefined terms were identified. Second, we checked the abstract and the full text for relevance. We summarize our results for each keyword combination in the respective database in Table 2. The keyword and backward search yielded 117 final hits.

Table 2 Summary of literature review findings
Full size table

For papers to be considered as a final hit, it is essential that they include AR or VR applications, whether in the form of a user study, a proof of concept, or a discussion about AR or VR utilizations in businesses. Papers about AR and VR applications that enhance humans in daily issues were excluded as irrelevant. Articles must have an apparent reference to workforce training in order to be included. For instance, research investigates the learning and skill development behavior of college students of an undergraduate building materials course with the aim “to close the skills gap in [the] construction industry” (Wu et al. 2018, p. 4). As this study could be the basis for training programs in construction or engineering, this paper was deemed appropriate for our research. The same holds for articles with ideas that are applicable to training processes to enhance workers abilities [e.g., Li et al. (2010); Lafreniere et al. (2016)].

When conducting the backward search, we analyzed the potential results regarding AR and VR, as described earlier. Due to their similarity, results in other technical disciplines such as mixed reality or other human–computer interfaces appeared. If they were sufficient to enhance humans’ capabilities in their job, we also included them in our research. During the backward search, several references appeared more than once (duplicates) which can be considered as an indicator for the saturation of a review (Boell and Cecez-Kecmanovic 2014).

We analyzed all hits concerning their application to workforce training and support and the rendering technology used for AR and VR. Three types of AR devices can be used to structure the results, including head-mounted displays (HMD) like the Microsoft HoloLens (Microsoft 2017), handheld devices and projectors (Nee et al. 2012). Within the field of VR, we focus on an immersive virtual experience by using VR headsets. Haptic devices, which involve physical contact between the user and the computer by sensing the movements of the human body, can enhance the sense of realism in a virtual environment by providing realistic feedback (Nee et al. 2012). Thus, we differentiate between the use of VR headset only and VR headset in combination with haptic devices.

Overall, we identified 14 articles focusing on workforce training for onboarding systems of which six are using AR. This system refers to training workers in new jobs without any prior experience. Thirteen articles examine how to upskill workers through on-the-job training, which involves hands-on practice of tasks respective to the job. This is done with HMDs (e.g., Acosta 2019) or AR projectors (Korn et al. 2015). Another 36 articles focus on both onboarding and upskilling training of which 23 articles focus on VR, or VR in combination with controllers. Thirty-five papers discuss “on-the-job guidance,” more precisely how to support workers in actively carrying out their job tasks. Almost 50% of these articles (17 out of 35) focus on HMDs to provide on-the-job guidance. Two of those 35 articles do not have a specified AR or VR technology in focus (Boud et al. 1998; Belani 2020). Support in “design and analysis” is covered by seven results, and further twelve results mention and compare various training purposes.

3.2 Practical research

Recently, IA in the form of AR and VR has gained a high degree of practical relevance due to applications in companies that have not been covered by theoretical search. Therefore, it is essential to conduct a practical search. To identify the hits among our findings, we employed identical criteria as in the academic search. To gain a comprehensive overview of AR applications, we are searching for use cases in the various occupational segments from Sect. 2.3. More precisely, we only consider the segments that show an increase in automation potential from 2013 and 2016 and have a medium or high proportion of potentially substitutable job activities in 2016. A medium substitution potential means that 30–70% of activities can be replaced, while a high potential refers to jobs with more than 70% automation potential of tasks (Dengler and Matthes 2015). For these segments, we expect highly diminished work activities, which can lead to the essential need for workers to change their professions or further training of their skills. The segments Security Professions and Social and Cultural Services professions can be excluded due to their low substitution potential. Although Medical and Non-medical Health professions do not reach the determined level of substitution potential, theoretical research has shown that VR training is becoming increasingly prominent in surgery, and we therefore we do include this segment in our practical search.

As a result, the following keywords together with Augmented Reality were used in our practical search on Google: Product Engineering; Manufacturing; Managers and Executives; Logistics and Transport; Logistics; Customer Interaction; Sales; Construction; Building; Food and Hospitality; Cleaning. The keywords Augmented Reality Sales Customer Interaction did not lead to any relevant use cases, which is why we added the broader keyword combination Augmented Reality Retail. For the occupational segment Business-Related Service professions, we selected three keyword combinations that correspond to the main fields of tasks of that segment: Augmented Reality Controlling, Augmented Reality Legal Services and Augmented Reality Financial Services. On average, the keyword combination for the respective occupational segment showed 20,092,000 results; we analyzed the results of the first page for each keyword combination, as the relevance declined drastically on the following pages. For the keywords Augmented Reality Agriculture; Customer Interaction; Sales and Food and Hospitality, we could not identify suitable use cases. In those segments, AR and VR applications are employed to enhance the customer experience through experiential marketing, such as IKEA’s AR app, enabling customers to preview furniture in their homes (Steuerwald 2019). Thus, AR and VR do not enhance workers in their job tasks. In addition, Augmented Reality Controlling and Augmented Reality Legal Services did not lead to any use case applications either. In total, we identified 30 use cases and three potential use cases (Table 3). The latter have not been tested in businesses yet; however, they outline promising approaches for interesting AR applications in businesses. We briefly discuss each field of application with selected examples. The full list of practical use cases is illustrated in Table 6 (“Appendix”).

Table 3 Summary of the practical search findings (*potential use cases)
Full size table

Six use cases provide VR onboarding training for non-specialist staff, defined as workers who do not have any previous experience in a specific profession. Deutsche Bahn (2018), DHL (2019) and UPS (2019) incorporated immersive VR into their training program at selected locations (Immerse 2019). DHL uses a virtual training program to coach new employees in the optimal packing of cargo space. A multi-user solution facilitates workers worldwide to train together in the virtual space. Caterpillar (2019) and Enel (2017) use VR headsets combined with controllers for employee safety training. The presentation of scenarios in plant maintenance allows experienced as well as non-experienced technicians to practice their tasks in virtual job scenarios, which are otherwise characterized by significant risks.

Moreover, the VR inclusion experience by DDI (2019), a global leadership consulting firm, showed that not only physical tasks but also social skills can be trained with VR. This use case immerses executives in a conference scenario, where they are confronted with inclusive and exclusive employees’ behavior. According to DDI, managers and executives develop a greater awareness of these behaviors, resulting in better employment management.

Another four use cases for workforce training aim to upskill employees with previous experience rather than training new employees. For example, Bosch uses the Common Augmented Reality Platform (CAP) in its service training centers in Germany and Denmark to train technicians in intricate repair work for vehicles. A field study showed that the use of the system results in a time saving of 15% on average in repair processes (Robert Bosch Gmbh 2019). One AR use case was found in Business-Related Service professions. The start-up 8ninths (2016) developed, together with Citibank, a proof of concept that uses AR in financial trading to visualize complex market data.

Besides on-the-job training, AR can leverage employees’ abilities by providing real-time support in operational tasks by displaying visual information into a physical job environment (Janin et al. 1993), which we refer to as “on-the-job guidance.” Five use cases are attributed to manufacturing and production occupations, in which AR supports workers with tasks in maintenance, repair and assembly. For example, Boing (2018) utilizes smart glasses to guide technicians through complex wiring tasks, with AR improving the time efficiency of wiring tasks by 30%.

In Transport and Logistics Professions, on-the-job guidance refers to improving the order picking process. AR systems are used to “offer[s] real-time object recognition, barcode reading, indoor navigation and seamless integration of information with the Warehouse Management System” (Glockner et al. 2014, p. 13). We refer to this method as “vision picking.” Coca-Cola and DHL incorporated vision picking within their warehouse operations equipping their employees the Microsoft HoloLens. According to Simon Woodward, the head of accelerated innovation and digitalization at DHL Supply Chain UK and Ireland, vision picking improves the productivity of warehouse operation by an average of 15% due to time efficiencies and error reduction (Williams 2019).

The same approach is also applied to the occupational segment Cleaning Professions, in which instructions are aligned to individual buildings and interior design. Records of cleaning processes and analyses of results can provide the user with regular feedback and can thereby accelerate the learning process (AR Check 2020). Seven use cases incorporate AR in design and analytical processes to allow workers to present their design ideas in a three-dimensional human scale and modify it further. Ford (2017) and NASA incorporated AR technology rendered through the Microsoft HoloLens, to previsualize their designs and prototypes, reducing technicians and time needed for the design processes (Engadget 2019). Furthermore, Ford is experimenting to teach their designers how to create three-dimensional cars in virtual reality. According to the Ford design manager Michael Smith, using the VR software Gravity Sketch reduces the design process of a car from several months to 40 h (Ford 2019).

We find five use cases in Building and Construction in which collaborative design is utilized to visualize digital mock-ups that can be “modified on the fly” by architects, engineers and customers simultaneously (Smith Group 2018). To this end, Smith Group and Gilbane Building Company have adopted the Microsoft HoloLens as an interface (Woyke 2019), while MLM Group creates three-dimensional models with the Wakingapp (2019).

Moreover, HMDs are also tested in Medical and Non-Medical Health professions. For example, SentiAR (2018) is working on displaying real-time holographic visualizations of patients’ hearts during cardiac surgeries. In the future, it can support surgeons in identifying the exact position of catheters and ablations.

The one potential use case in the Transport and Logistics sector involves the research project of the Fraunhofer Institute for Graphic Data Processing on processing shipping traffic data. Their aim is to provide skippers with all relevant data, which is displayed on “smart windows” to support foresight planning and decision making without the need to wear HMDs (Freiherr von Lukas et al. 2019).

3.3 Synthesis of theoretical and practical research

We use the concept-centric approach to synthesize the results of our theoretical (Table 4) and practical (Table 5) research (Webster and Watson 2002). The 117 theoretical and 33 practical results are listed based on their areas of application and the technical realization. The gray cells indicate that no paper or practical use case could be found for the respective area of application.

Table 4 Research papers aligned to areas of application
Full size table
Table 5 Practical research results aligned to areas of application (*potential use cases)
Full size table

On-the-job training is particularly effective for complex tasks that require a high degree of independence (Ponder et al. 2003). However, this requires the use of real machinery, which can limit the number of training participants, disturb daily business processes and result in inefficiencies (Lai 2019). These issues can be addressed by immersive VR (Jose et al. 2015). Virtual reality training allows users to repeat training scenarios multiple times to help retain their gained knowledge and increase their capacities (Mostafa et al. 2017). Thus, this training method can be applied for onboarding and upskilling purposes. Even though users benefit from immersive virtual environments by being able to engage in any situation (Ponder et al. 2003), to provide a realistic training experience, virtual training in physical tasks requires sensory feedback. Thus, several sources examine the advantage of a haptic component as to complement VR headsets in workforce training. According to the literature review, VR headsets and controllers are suitable for training novice users as well as for upskilling experienced workers. Safety training in particular reflects a successful area of application for VR training with its advantage of preventing accidents in real-life training (Zhao et al. 2009). The success of this training method is verified by various practical implementations in businesses as well as proofs-of-concept for mining training (Van Wyk and De Villiers 2008).

In contrast to VR, AR in onboarding training has not yet been incorporated into business. However, theory suggests that AR applications would enhance the traditional learning process by overlaying augmented information or graphics on books through the use of smartphones or HMDs (Behzadan et al. 2011). Augmented reality in the form of HMDs allows internal training sessions in which trainer and trainee are not obliged to be at the same location. In contrast, when they share the same location, trainees and trainer can practice in their own virtual space, which allows an individualized training program (Funk et al. 2017).

Our results show that AR particularly useful for real-time on-the-job guidance, as an increase in product variations, triggered by a great level of customization, requires unique assembly and maintenance instructions (Thomas and Sandor 2009). Several businesses apply AR in form of smart glasses for order picking, assembly and maintenance processes, which result in time savings (Büttner et al. 2017b). Our theoretical results indicate that AR is highly relevant due to its various applications in repetitive, well-defined, procedural tasks as long as instructions are adapted to the learned skills of the user (Wilschut et al. 2019).

In sum, the concept matrix presents the most popular ways to support the workforce with AR, namely, on-the-job guidance and collaborative design and analysis. However, it must be noted that some results for on-the-job guidance are closely related to the upskilling phase. Furthermore, the table shows that HMDs are the most popular technology for rendering AR in theoretical user studies and proofs-of-concept, as well as in practical applications. While AR projectors are theoretically investigated, handheld devices show a greater application in businesses. VR is mainly used in training contexts with a focus on using VR headsets in combination with controllers.

4 A framework for intelligence augmentation solutions

Our previous findings inform the development of a framework aiming to give instructions as to how AR or VR can support workers at their job so that they can keep up with changing demands. Furthermore, it aims to show for which application areas AR or VR can provide workers with sufficient training to learn new job tasks. The previous literature review as well as the practical use cases have shown that different applications of AR and VR can be assigned to three main task areas: Procedural Activities, Analytical and Creative Activities, and Social Activities. Thus, we map the job segments (cf. Table 1) to the aforementioned task areas. All job segments that involved a high proportion of human interaction tasks were assigned to Social Activities. Procedural Activities included job segments with well-defined and mostly repetitive tasks, whereas Analytical and Creative Activities contained jobs that required designing, decision-making and logical reasoning skills. Figure 1 illustrates our overall framework, which we exemplarily discuss for hypothetical applicational scenarios in the subsequent sections.

Fig. 1
figure 1

A framework for using AR and VR in workforce training (images are own representations)

Full size image

The framework is structured as follows: The first step in applying AR or VR to workforce enhancement is to identify the scope of application. Second, the objective of applying AR or VR in the respective activity field must be specified. Third, depending on the defined training objective, the decision about the theory of learning follows. This learning theory refers to the method by which the worker acquires new skills. We derived four theories of learning based on the synthesis of the theoretical and practical findings: immerse learning, instructive guidance, collaborative support and visualization. After selecting the learning method, the framework can be used to determine the technological devices to finally arrive at the respective training activity.

4.1 Procedural activities

The application area of Procedural Activities corresponds to clearly defined sub-processes that leave limited opportunities for creativity. The activities refer to sequences of repetitive physical or mental activities. Consequently, individuals can learn them without the need for prior experience in the job. Jobs with logistical, mechanical, assembly, cleaning and cooking activities can be assigned to this area of application. When employees aim to acquire new skills or procedures in the latter jobs, workers can practice them through immersive learning. This learning method is most efficiently carried out with VR since it makes learning more interactive and tangible, as the user is immersed in the work scenario. Compared to training in real-world scenarios, the virtual environment offers the advantage of errors not having any severe consequences and allowing workers to practice new procedures. The interface between humans and haptic devices in the simulation must be similar to the interface in real life; for example, the task of tightening a screw should not be replicated with a mouse click in VR.

Another purpose of workforce enhancement for procedural activities is to support employees’ in current activities. This goal is initiated with the learning theory “instructive guidance,” whereby workers receive support while carrying out their job tasks. Hence, individuals can carry out a task without comprehensive previous knowledge or experience in that field. In this approach, AR applications are used to display real-time information about the employee’s work environment, allowing her to accelerate the respective task process and learn new procedures.

Even though it is conceivable that instructive guidance could be used for training new job capabilities and supporting workers in their current capabilities on-the-job, our findings from the theoretical and practical search primarily identified on-the-job support as a significant application area for instructive guidance. Therefore, we intentionally decided to use this theory of learning only for scenarios aiming to leverage worker’s current job capabilities.

For example, due to the ethical dilemma of military robots (Lin et al. 2008), military occupations are less likely to be automated, and hence, tasks such as shooting continue to be performed by humans. Soldiers are equipped with an VR headset with a motion tracker and a haptic device, which is shaped identically to real military weapons. The monitoring of shooting activities can be easily replicated in the virtual environment, since pressing the button of the dummy device is similar to pulling the trigger on a gun if only the movement itself is being judged. Therefore, VR training offers an almost realistic alternative to real-life processes.

4.2 Analytical and creative activities

Analytical and Creative Activities are characterized by a high demand for expertise in a specific job environment. These tasks require mainly critical and creative thinking to meet challenges of designing, engineering, decision-making processes, data or technical analyses and surgery. These activities are subject to a high degree of variability and often require tertiary education. Thus, they cannot quickly be learned through on-the-job training without any previous knowledge in the field. Therefore, the fundamental purpose of applying AR or VR in this activity area is to expand employees’ scope of action in their current job. Some of the mentioned activities involve various participants and demand collaborative support. To this end, AR rendered by HMDs has become apparent as the best solution to provide such support. When multiple workers from different departments meet to discuss the next phase of a process they are working on, they are equipped with AR glasses, which allow participants to simultaneously access three-dimensional visualizations of data and mock-ups, to overlay these digital elements on real-world objects and to make modifications.

In this approach, AR plays an essential role in facilitating interactive working by allowing distributed teams to work together in real time. Moreover, a significant added value of AR use is the visualization of information, data and designs.

An exemplary application of AR in Analytical and Creative Activities would be the consulting industry. Most tasks of consultants include data analysis and presentation. Furthermore, this job is characterized by high geographical flexibility. To support consultants in their daily job activities, they are equipped with smart glasses, which can receive information from the customer conversation via the voice control function. These clients' statements are filtered according to their relevance for the subsequent discussion. For example, the question of whether investment in a medium-sized company is profitable will be streamed to an off-site server. This server collects publicly available data as well as data from an in-house database. The collected data packets are analyzed and prioritized. As the system is dynamic, it can send all relevant data packages back to the consultants’ display in the glasses. The consultant then reviews the data to provide real-time advice to the client based on this information.

4.3 Social activities

Social Activities refer to jobs with a high demand of human interaction tasks, which require constant adaptation to changing circumstances. However, these jobs also require documentation of data, for example, customer data. Therefore, leveraging workers in their job environment is achieved by providing them with information about customers. Since continuous communication is especially important in direct customer contact (Gouthier et al. 2007), this information must be displayed in real time in the user’s real environment. Augmented reality can provide optimal support in this scenario. All relevant data are stored on a platform, which can be accessed and displayed on smart glasses.

When training workers in new job capabilities in social activities, immersive VR training can be used following the same rationale as for procedural activities. However, it is essential to mention that social activities require a different organization of the training program than the predictable procedural activities. They are acquired through observational learning, modeling and feedback (Elliott and Gresham 1993). Therefore, in our framework, users are immersed in a virtual scenario in which they are passive spectators to the displayed situation rather than actively carrying out a task.

Research shows that teaching behavior and professional and social actions can be enhanced by video recordings. Furthermore, the observation one's own behavior increases the ability to recognize the positive or negative effects that arise in social interaction with other human beings (Brouwer 2014). Hence, the virtual scenarios that are displayed to the trainee correspond to recordings of real work situations, for example, a business meeting or a teaching lesson. In such a scenario, avatars replicate the behavior of people in real-world work situations so that individuals can relate to these behavior patterns. This approach has the benefit of giving users a sense of presence in the displayed work scenario, allowing them to actively experience such a situation as a participant rather than passively watching a two-dimensional video recording (Riva et al. 2007). This method allows the trainee to compare his or her perception of their own social abilities with behavioral patterns observed from the “outside.”

An AR application for Social Activities could be found in the hotel industry. Booking details and further information on former guests can be stored centrally on a server, and receptionists can be equipped with smart glasses that recognize a guest when entering the hotel. All relevant data about the visitor is sent from the server to the smart glasses, allowing the receptionist to welcome the guest on a more personal level. Moreover, data can be added via the voice recognition function, thereby eliminating time-consuming administrative work. By displaying all customer information in real time into the receptionist’s field of vison, the latter can respond instantly to customer requests, which can result in higher customer satisfaction (Davis 1991). Robotic staff or digital check-in counters can offer the same advantage without any human interaction. However, it must be noted that this was implemented in the daily hotel business of the Henn na Hotel in Japan, and these robots created extra work for both staff and guests due to their poorly developed technology (Hertzfeld 2019). Moreover, service failure due to the missing human interaction component (Girman et al. 2009) can lead to customer dissatisfaction and therefore lower customer loyalty (Parks 2010).

5 Conclusion and outlook

We analyzed how AR and VR can help individuals’ transition into a new occupation or provide them with a new skill set required in the emerging job landscape. We found that AR is most suited to guide workers on-the-job, mainly with the help of HMDs in the context of well-defined procedural tasks. In contrast, VR, in the form of VR headsets with controllers or other haptic devices, is mostly used to provide workers with onboarding training as well as to upskill workers and employees. Moreover, the results reveal that VR enables workers to collect new skills by repetitive training in their respective work tasks.

The conceptual framework developed in this work highlights the appropriate application of the mentioned technologies in workforce training. Through the framework, it becomes clear that the activities of the segments Manufacturing and Production; Transport and Logistics; Agriculture, Forestry, and Horticulture; Food and Hospitality; Security Professions; and Cleaning Professions can be trained by immersive VR and efficiently conducted by getting AR support. This enables employees to keep up their skills for a unique assembly process due to the increasing product variation caused by the trend toward a high degree of customization. Furthermore, the framework reveals that the application of AR allows collaborative work and three-dimensional visualizations for analytical and creative tasks in the occupational segments Managers and Executives, IT and Scientific Services, Business-related Services, and Building and Construction. Thus, in these segments, AR enhances human capabilities in activities that have a high degree of variability and will therefore be of even greater importance in the future. Another outcome of the framework shows the suitable application of AR and VR in social activities, including jobs in the occupational segments Social and Cultural Professions, Customer Interaction and Sales, and Food and Hospitality. Individuals can improve their social skills through VR training, while AR provides the opportunity to focus on human interaction activities in these work fields. AR is able to leverage human capabilities by optimizing the work behavior through real-time instruction and in tasks. This allows workers to concentrate on using more cognitive capabilities. However, the different AR applications are not used to the same extend in companies. So far, HMDs have primarily been used in companies, even though projectors are a promising alternative. The theoretical results on AR projectors can, however, serve as a basis for companies to implement projectors in the future.

Moreover, VR supports workers in changing occupations with the help of an immersive training system. The VR training process is supported by haptic devices that are not yet technically matured to enable reliable training for precise tasks, such as in surgical training. In this context, it would be interesting to conduct further research on the aforementioned problem, to improve and adjust the accuracy of the technology and guarantee error-free surgeries. Haptic devices without the use of VR, have also demonstrated to be an efficient IA technology, and we, therefore, suggest further investigation of haptic guidance.

Prospectively, the further development of the VR and AR training systems is of utmost importance, as we have shown that they are helpful instruments to extend the employees' scope of action in their respective work tasks but are still in the early implementation phase in companies. IA technologies would have the potential to prepare employees for significant changes in the employment landscape in Germany, if companies are made aware of the possibilities of these technologies. Thus, we argue that IA technologies can help especially workers in jobs with lower education in transitioning in the next years and thereby help alleviating the unforeseeable social explosive force which lies within this technological change.