Monitors vs. Smart Glasses: A Study on Cognitive Workload of Digital Information Systems on Forklift Trucks

Abstract

New digital work equipment is increasingly affecting the approach to work. They enable more efficient, mobile and flexible operation, at the same time, however, new potential hazards and risks may arise for employees. A number of digital devices and their corresponding application context have not yet been comprehensively researched in the field of occupational sciences. In order to be able to recognize potential hazards in a preventive approach, this study examines the impact of the use of such devices in the context of forklift trucks on the safety and health of employees. In particular, the use of two types of smart glasses and of a monitor has been investigated. For the assessment of cognitive workload, the study made use of a standardized reaction test procedure, the Detection Response Task (DRT) according to ISO 17488 as well as subjective assessment using the NASA Task Load Index (NASA-TLX). This research will provide valuable information about possible work related hazards regarding the use of smart glasses on forklift trucks with respect to cognitive workload.

Keywords

1 Introduction

1.1 Smart Glasses

Smart glasses are head-mounted micro-computers (wearables) that have optical displays in the user’s field of vision. Therefore, these devices are classified as head-mounted displays (HMD). Similar to smartphones, they have an operating system, usually Bluetooth- and WLAN-sensors as well as a camera, with which videos and photos can be recorded and displayed. By using smart glasses, reality is enriched by additional virtual overlays of information or objects in real time. Therefore, in the context of smart glasses we are referring to assisted reality or augmented reality.

Potential use cases can be found in all areas of work: Tasks that require guidance, information recording, collaboration or external control, but where visual obstruction or interaction with an external display is impeding. Advantages include, for example, the reduction of training requirements through context-related work instructions that are displayed in the employee’s visual field with smart glasses (training on the job). Or for other applications, assistance and support (remote expert) can also be consulted for specific operations, or the work quality can be checked via video stream and documentation [1].

1.2 Smart Glasses in Warehouse Logistics

In recent years, the potential of smart glasses and augmented reality has met with great interest in different areas of the working world, especially in warehouse logistics. This service sector with its complex work processes, high demands on mobility and flexibility is particularly affected by technological transformation. Today, order picking scenarios can already be implemented by using smart glasses in the form of pick-by-vision systems: Hands-free operation, integrated barcode scanners, and the reduction of glances on external information devices when working with smart glasses provide significant advantages over conventional picking systems. Therefore, the US market research company Gartner expects that 20% of large enterprise businesses companies will start implementing virtual or augmented reality solutions by 2019 [2].

Although recent operational scenarios of smart glasses have mostly been limited to picking activities, it is also conceivable that such devices can be used as digital information systems for operations forklift trucks. In general, forklift truck drivers receive work related information via monitor display systems in the passenger compartment. In 2016 alone, the German Social Accident Insurance (DGUV) reported 218 213 industrial accidents in internal transport, the most frequent of which were accidents involving industrial trucks (forklifts, hand trucks) and 12 671 of which with forklifts [3]. The most common cause of accidents is the driver’s misconduct or inadequate operational safety precautions. And while there are no exact figures on distraction related accidents, corresponding studies on information and communication technology in road traffic show that reducing the number of non-driving activities can make a fundamental contribution to accident prevention [4]. In this respect, the driver’s workplace should be designed in such a way that the cognitive workload is minimized [5]. In addition, long-term effects in the musculoskeletal system due to the use of conventional and head-mounted displays are also conceivable.

With smart glasses, information is presented to the user in the ego-perspective, which may lead to a reduction of glance related distraction from the driving activity. Therefore, it is essential to examine whether this positive aspect corresponds with a reduction of cognitive workload.

2 Leading Question

The study examines whether the use of display systems while driving forklifts is related to an increase in cognitive workload, and to what extent the use of smart glasses on forklifts differs from conventional monitoring systems.

3 Methods and Materials

3.1 Forklift Truck Simulator

For the study, the forklift truck simulator of the Institute for Occupational Health and Safety (IFA) was used, which simulates the original working environment and a vehicle with a fully haptic passenger compartment (see Fig. 1). Standardized mounts for mobile devices have been attached to the center console to enable display-driven interaction with external control panels.

Fig. 1.

Forklift truck simulator in the experimental setup.

The simulation software (Unity Engine) was presented on three 55-inch flat screen full HD monitors. Depending on the settings of the position-adjustable driver’s seat, the eye to display distance varied from 85–100 cm.

Both, the technical structures and their arrangements were based on a maximum achievable level of realism regarding operational practice.

3.2 Digital Information Systems

Three different digital information systems were used: As a representation of a monitor system, a 10.1-inch tablet (tablet) was used, which was attached to the center console with a bracket for the duration of the respective runs. For the runs with head-mounted displays, two different smart glasses were used: A smart glass (head-mounted display 1: HMD1) with a monocular display and touchpad on the side frame. The second smart glass (head-mounted display 2: HMD2) has a binocular display system and an external touchpad connected to the frame with a wire and mounted in the center console.

3.3 Participants

Only persons with a valid forklift license were included as test persons in the study collective. In addition, the drivers were not allowed to wear glasses or at least had to be able to drive a forklift truck without glasses. This restriction was necessary because the head-mounted display devices could not be fully used with glasses. Prior to the test, the participants were informed about the general conditions of the study. All persons participated on a voluntary basis after giving informed consent in written form. It has been possible to terminate the test at any point without negative consequences for the participants.

A total of 32 male participants of the IFA and the cooperating partner, a company for warehousing and logistics, were invited to take part in the study. Data from nine participants were not evaluated because there were cases of simulator sickness or incomplete data sets. The remaining 23 participants had an age between 23 and 53 years with a mean value of 40 years. One of the subjects had glasses, but took part in the experiment without glasses. The other 22 subjects were not wearing glasses.

According to the participants, the driving experience with forklifts was between 0.3 h and 40.0 h per month with a mean value of 6.5 h and a median of 3.0 h.

3.4 Tasks

The three tasks according to ISO 17488 [6] were implemented as follows:

The primary task was to operate in the driving simulator, which displayed a virtual warehouse without other road users with numbered shelf paths. In this warehouse, shelves with certain numbers had to be approached, and driving errors were also recorded in every run. A driving error occurred when objects (walls, shelves) were touched by the vehicle in the simulation.

The secondary task consisted of reading information from three different types of outlined digital information systems. The participants were instructed to read a randomly selected shelf number and approach the corresponding target (see Fig. 2). This task was repeated until the end of the respective run. In addition, the task load has been held on the same level by constantly reading out numbers (1–10) appearing on the respective display. Each participant completed a total of four runs in random order, each lasting two minutes: One run each with one of the two smart glasses, the tablet and a run without additional tasks. This baseline run consisted of driving without display units, whereby the shelf numbers to be approached were read aloud by the test supervisor.

Fig. 2.

Secondary task performed with smart glasses (HMD1).

The reaction test was carried out according to the standard with an optical stimulus using the Detection Response Task (DRT). The light emitting diode was located in the participants’ field of view. For each run, participants had to react to about 30 stimuli by pressing a button mounted to their index finger. A hit was counted and stored with the corresponding response time if a reaction took place between 100 ms and 2500 ms. In addition to the response time, the hit rate was also recorded.

3.5 Questionnaires

Before each session, participants were asked to complete a questionnaire with demographic information. It included questions on the following aspects:

• age (in years)

• gender

• wearer of glasses (yes/no)

• driving experience (in hours per month)

In addition, the subjective load for the respective devices was reported by the participants after each run by means of a standardized questionnaire (NASA-TLX) [7]. It consisted of six individual questions on different aspects of the load, which were answered by placing a cross on a scale. Since the scale contained 20 tick marks between “low/good” and “high/bad”, the crosses were evaluated as numbers between 0 and 20. The individual answers were not weighted.

After the sessions, general questions were asked about which system was suited best, which one was least distracting, and whether a certain device obstructed the view.

3.6 Procedure

Before each session, participants were asked to carry out training runs without any secondary tasks until they felt comfortable with the handling of the simulation. Thereafter the secondary task, the reaction task as well as the respective display units were introduced to the participants. Subsequently, a further practice phase was carried out, the procedure of which corresponded to the real test without a reaction test and questionnaire. This was followed by 4 runs, during which the test persons had to perform the DRT additionally:

• run without display unit (baseline)

• run with monocular display unit (HMD1)

• run with binocular display (HMD2)

• run with an external monitor (tablet)

The order of the runs was determined randomly. Overall, one session lasted approximately 30 min.

4 Results

Three performance parameters from the objective measurements are used in the experimental setup: the driving errors, the response times of the hits and the hit rates. The results of the objective measurement are accompanied by a standardized questionnaire, the NASA-TLX. The general questionnaire can only provide supporting information.

4.1 Driving Errors

The analysis of driving errors for entirety of the runs by the 23 participants shows a small number of driving errors: only in 21 out of 92 complete runs driving errors occurred. When adding up the errors during the runs regarding the corresponding digital information systems, four driving errors were recorded for the baseline run and 12 driving errors each for HMD2 and tablet. However, there were only five driving errors when using HMD1. It has not been possible to infer with statistical methods a dependence of the driving errors on the participants and their cognitive task load. The driving errors appear to happen randomly on a low level.

4.2 Response Times and Hit Rates

The study examined, whether the cognitive workload increases when using digital information systems compared to baseline runs and whether the workload of the evaluated devices differ from one another. The results show that mean response times per participant with digital information systems were lower compared to runs without additional tasks (see Fig. 3): For runs with HMD1, the differences in response times per participant were on aver-age 147 ms lower, with HMD2 130 ms and with the tablet 103 ms. This effect is statistically significant.

Fig. 3.

Distribution of the differences of mean reaction times per participant as box plots and scatters plots. Mean reaction times while using a device is subtracted from runs without device, i.e., negative values indicate an increase in cognitive load.

In addition, the hit rate decreases on average per participant when digital information systems are used in the runs.

However, this effect cannot be observed among the devices examined: There was no significant distinction in response times or hit rates for the use of one device compared to another. It should, therefore, be noted that the cognitive workload during runs was lower if no additional tasks had to be carried out on digital information systems. But none of the examined devices produced a higher or lower cognitive workload in direct comparison of the devices.

4.3 Questionnaires

The results of the NASA-TLX support the findings of the objective measurement: If comparing the results of the NASA-TLX for runs without devices and runs with digital information systems, the analysis shows that the subjective workload when using the devices increases. In addition, if one compares the differences between different devices, the analysis does not show any significant difference in the workload.

When it comes to the results of the general questionnaire, most of the respondents liked the tablet best, HMD1 came in second, and only two respondents preferred the HMD2 display. This was also the case when referring to the least distracting device: only two respondents gave a different answer to the question of the preferred device in this regard. The issue of visual obstruction implied the same understanding as the question of least distracting device: Overall, the tablet was associated with the least distraction and least general obstruction, the HMD2 display with the largest.

5 Discussion

5.1 Participants

Participants in the study were selected in advance according to specific criteria: they had to be employed either by the partner company or at the IFA, and they were not allowed to wear glasses.

They also had to hold a valid forklift truck license. However, it can be assumed that forklift truck drivers generally have a higher monthly driving experience than the ones reported by the participants of this study. Therefore, the study collective is not representative and the results of the study can only be applied to the group of all forklift truck drivers with limitations.

5.2 Driving Errors

Since the driving errors occur in this test at a low rate, that appears to be independent from the participants and their workload, the primary task of driving in the simulator has been successfully accomplished regardless of the comparatively low reported driving experience.

On the other hand, one cannot infer from this that driving errors are generally independent from personal training level and/or workload. However, this has to be investigated in a different test design.

5.3 Response Times and Hit Rates

The results of the objective measurement show that the cognitive workload on the participants has increased significantly due to the use of display devices (HMD1, HMD2, and tablet).

Furthermore, it was not possible to clarify whether there is a difference between the digital information systems regarding cognitive workload. Neither the response times nor the hit rates had conclusive or significant effects that allow a specific device to be prioritized. Therefore, the question of the differences between the devices must be clarified in subsequent studies.

5.4 Questionnaires

The analysis of the exposure aspects in the NASA-TLX questionnaire has been inconclusive. Thus, a weighting of these aspects in the analysis is not possible, and only a mean value of all six questions has been analyzed. This quantity is more related to a general, subjectively felt workload than a measurable cognitive workload.

However, it is plausible that the results of the NASA-TLX strongly correlate with the cognitive workload of the outcome of the objective measurements.

Despite the NASA-TLX’s lower validity compared to the objective measurements, the analysis delivers the same qualitative results: Generally, workload increases significantly with the use of devices, but no difference in workload between the devices can be determined.

The results of the general survey were the least reliable. The preference for a certain device can be influenced by various factors and therefore indicates only a partial connection to the issue of cognitive workload. Nevertheless, it is not unlikely that the preference for the tablet is due to the fact that the participants felt least distracted by this device. Although no objective difference in workload has been observed in this study, it can be assumed that the acceptance of the tablet is higher than for the head-mounted display devices.

6 Conclusion

The study investigated cognitive workload caused by the use of digital information systems on forklift trucks during driving. The results provide information on the cognitive workload potential of the investigated devices types: monitor monocular and binocular smart glasses.

Regarding the key study issue, it has been found that the cognitive workload on drivers when using display information devices increases significantly. This was verified both by the results of the objective measurements according to ISO 17488 and by the subjectively perceived workload according to the NAS-TLX questionnaire. However, the results of the study do not reveal any significant differences in cognitive workload between the three digital information systems. Therefore, the question of the differences between the devices must be clarified in subsequent studies. However, the use of a monitor as a digital information system has the distinct advantage over smart glasses that it can also be used by spectacle wearers. In addition, the study has shown that the acceptance of this device is probably higher than with head-mounted display systems.

In addition, it is unclear whether the workload determined in the study is associated with physical strain. The use of smart glasses in this work scenario may, for example, cause musculoskeletal problems that could become chronic if exposed for a longer period of time. It is also possible that the use of smart glasses will increase the risk of accidents caused by distraction. This would have to be clarified in further studies.

The impact of using smart glasses on forklift trucks in a real-world working environment cannot be fully covered by the study. It would be substantially influenced by factors that have to be assessed in the context of the respective risk assessment: type and complexity of tasks to be performed, duration of use, and the software employed. Before using Smart glasses on forklift trucks, companies should therefore consider whether an additional workload is acceptable.