3.1 Forklift Truck Simulator
For the study, the forklift truck simulator of the Institute for Occupational Health and Safety (IFA) was used, which simulates the original working environment and a vehicle with a fully haptic passenger compartment (see Fig. 1). Standardized mounts for mobile devices have been attached to the center console to enable display-driven interaction with external control panels.
The simulation software (Unity Engine) was presented on three 55-inch flat screen full HD monitors. Depending on the settings of the position-adjustable driver’s seat, the eye to display distance varied from 85–100 cm.
Both, the technical structures and their arrangements were based on a maximum achievable level of realism regarding operational practice.
3.2 Digital Information Systems
Three different digital information systems were used: As a representation of a monitor system, a 10.1-inch tablet (tablet) was used, which was attached to the center console with a bracket for the duration of the respective runs. For the runs with head-mounted displays, two different smart glasses were used: A smart glass (head-mounted display 1: HMD1) with a monocular display and touchpad on the side frame. The second smart glass (head-mounted display 2: HMD2) has a binocular display system and an external touchpad connected to the frame with a wire and mounted in the center console.
Only persons with a valid forklift license were included as test persons in the study collective. In addition, the drivers were not allowed to wear glasses or at least had to be able to drive a forklift truck without glasses. This restriction was necessary because the head-mounted display devices could not be fully used with glasses. Prior to the test, the participants were informed about the general conditions of the study. All persons participated on a voluntary basis after giving informed consent in written form. It has been possible to terminate the test at any point without negative consequences for the participants.
A total of 32 male participants of the IFA and the cooperating partner, a company for warehousing and logistics, were invited to take part in the study. Data from nine participants were not evaluated because there were cases of simulator sickness or incomplete data sets. The remaining 23 participants had an age between 23 and 53 years with a mean value of 40 years. One of the subjects had glasses, but took part in the experiment without glasses. The other 22 subjects were not wearing glasses.
According to the participants, the driving experience with forklifts was between 0.3 h and 40.0 h per month with a mean value of 6.5 h and a median of 3.0 h.
The three tasks according to ISO 17488  were implemented as follows:
The primary task was to operate in the driving simulator, which displayed a virtual warehouse without other road users with numbered shelf paths. In this warehouse, shelves with certain numbers had to be approached, and driving errors were also recorded in every run. A driving error occurred when objects (walls, shelves) were touched by the vehicle in the simulation.
The secondary task consisted of reading information from three different types of outlined digital information systems. The participants were instructed to read a randomly selected shelf number and approach the corresponding target (see Fig. 2). This task was repeated until the end of the respective run. In addition, the task load has been held on the same level by constantly reading out numbers (1–10) appearing on the respective display. Each participant completed a total of four runs in random order, each lasting two minutes: One run each with one of the two smart glasses, the tablet and a run without additional tasks. This baseline run consisted of driving without display units, whereby the shelf numbers to be approached were read aloud by the test supervisor.
The reaction test was carried out according to the standard with an optical stimulus using the Detection Response Task (DRT). The light emitting diode was located in the participants’ field of view. For each run, participants had to react to about 30 stimuli by pressing a button mounted to their index finger. A hit was counted and stored with the corresponding response time if a reaction took place between 100 ms and 2500 ms. In addition to the response time, the hit rate was also recorded.
Before each session, participants were asked to complete a questionnaire with demographic information. It included questions on the following aspects:
In addition, the subjective load for the respective devices was reported by the participants after each run by means of a standardized questionnaire (NASA-TLX) . It consisted of six individual questions on different aspects of the load, which were answered by placing a cross on a scale. Since the scale contained 20 tick marks between “low/good” and “high/bad”, the crosses were evaluated as numbers between 0 and 20. The individual answers were not weighted.
After the sessions, general questions were asked about which system was suited best, which one was least distracting, and whether a certain device obstructed the view.
Before each session, participants were asked to carry out training runs without any secondary tasks until they felt comfortable with the handling of the simulation. Thereafter the secondary task, the reaction task as well as the respective display units were introduced to the participants. Subsequently, a further practice phase was carried out, the procedure of which corresponded to the real test without a reaction test and questionnaire. This was followed by 4 runs, during which the test persons had to perform the DRT additionally:
run without display unit (baseline)
run with monocular display unit (HMD1)
run with binocular display (HMD2)
run with an external monitor (tablet)
The order of the runs was determined randomly. Overall, one session lasted approximately 30 min.