1 Introduction

The mobile machine operator plays a significant role in optimizing the overall performance of a human–machine system. Human factors must therefore be closely considered in order to provide operators with the best possible conditions in which to perform efficiently. For example, limited visibility of the operating area can decrease operator performance and have a negative effect on productivity and the quality of the end product. According to Kuijt-Evers et al. (2003), wheel loader and excavator operators rated view and visibility as the second and third most important factors affecting their ability to work effectively from the cab. Limited visibility of the operating area surrounding the machine can endanger the safety of the operator and of other persons present within the machine’s operating area and reduce the performance efficiency of the task (CEN/TR 614-3 2010). Limited visibility can cause accidents such as collisions with other workers or vehicles, driving over an unseen edge or rollovers (Abubakar et al. 2010; Ruff et al. 2011). Kumar and Ghosh (2014) mentioned visibility as one of the main causative factors of accidents with open-cast coal mining machinery.

Traditionally, visibility and blind spots have been taken into consideration during the design phase by using light bulb shadow tests (ISO 5006 2006). In addition, novel approaches, such as digital human models (DHMs), have been used to evaluate visibility (Badler 1997). However, both of these methods are relatively static approaches and do not necessarily consider machine operator visibility as it relates to real task performance, especially if the DHM postures are formed without tracking real operators. The use of virtual environments (VEs) as a virtual prototyping (VP) system could provide an effective tool for designing visibility scenarios and solutions, as VE facilitates consideration of the operators’ task performance in a realistic context prior to realizing the actual machine cab.

The goal of this study was to investigate the following questions: ‘Does the transparency level of the boom affect operator performance?’ and ‘Does the placement of overlaid information affect operator performance?’ The structure of the paper is as follows: Sect. 2 presents related work, Sect. 3 describes the visibility context of this study, Sect. 4 illustrates the used methods and study design, Sect. 5 describes the results, Sect. 6 discusses the findings and Sect. 7 concludes the research findings and their implications and proposes directions for future research.

2 Related work

Visibility can be defined as the clarity with which an object in the field of view may be perceived (Barron et al. 2005). If this definition is turned around, it can be said that a blind spot is a distortion or absence of visualization of a portion of the visual field (Kabir et al. 2015). Such blinds spots can occur when a machine structure masks the operator’s field of view.

Visibility issues related to mobile machines have been addressed in many studies. According to Bhattacherya et al. (2006), the most commonly used visibility assessment techniques are line-of-sight shadowgraphy (e.g. light bulb shadow test) (ISO 5006 2006; Kabir et al. 2015), panorama photography and computer simulation programs (Badler 1997). In addition to these, Bhattacherya et al. (2006) proposed the use of laser scanning for collecting visibility information. Gilad and Byran (2015) used their developed field-of-view analysis methods based on DHM to measure the visibility from three different tractors. They evaluated visibility around the cab at a height of 30 cm. Lee et al. (2016) developed a new evaluation method for cab visibility. The method used viewpoints, both inside and outside, of Far East combine harvesters. Choi et al. (2009) studied the visibility of a forklift truck using the light bulb shadow test, a digital human model and real participants. They found that the evaluation with real participants provided better visibility results. Godwin and Eger (2009) tested three camera locations for their ability to increase the visibility of areas around a load-haul-dump mining vehicle that were very difficult to see from the operator’s position. All of the camera locations were shown to improve visibility. A VE-based task-related field-of-view analysis was proposed by Aromaa et al. (2012). The method can be used, for example, to compare two different machine cab structures. In addition, DHMs have been used to support the design of field-of-view issues (Choi et al. 2009; Di Gironimo et al. 2012; Godwin and Eger 2009; Godwin et al. 2008). Summerskill et al. (2016) studied blind spots in the driver’s view from large goods vehicles. They used DHMs for the identification and presentation of mirror and blind spot modelling.

The aforementioned evaluation methods have usually been employed when designing next-generation machines (e.g. in next-generation tractors the location of the exhaust pipe should be changed in order to achieve better visibility (Gilad and Byran 2015)).Traditionally, measures intended to improve visibility have included enlarging the window surface, changing the cab structure, use of mirrors and provision of additional camera views. In the future, another approach will be considered: see-through structures. See-through structures are usually created using direct or virtual image formation techniques. Tachi et al. (2014) presented a transparency idea for automobiles utilizing a retroreflective projection technology (RPT) system that reflects light back from retroreflective surfaces, making them appear transparent. Land Rover’s transparent bonnet concept also shows what see-through structures could mean for cars (Land Rover 2014). Another example, within the tractor industry, is Driver Extended Eyes by DeutzFahr (DEUTZ-FAHR (official) 2015), which displays occluding areas on a separate monitor.

See-through solutions and transparency issues have been studied in the design of computer displays. Lindlbauer et al. (2016) investigated the influence of display transparency on task performance in the context of a customer service situation. The use of a transparent display did not undermine task performance. Harrison et al. (1995) studied visual distinctiveness and levels of transparency in the design of user interfaces. They found that icon type, background image type and transparency all affected response time performance.

Another quite new technology, which is becoming mature enough to be implemented into the work machine context, is augmented reality (AR). AR means that the operator is able to see the real world, but with virtual objects superimposed upon or composited with it (Azuma et al. 2001). AR technologies have been applied especially in design and manufacturing (Nee et al. 2012; Ong et al. 2008; Wang et al. 2016). An important aspect in the AR context is the management of occlusion, i.e. the way that occluding and occluded objects are treated and visualized. Apart from simple video analysis-based approaches, as demonstrated by Lu et al. (2010) and Chen et al. (2010), the more sophisticated methods are based on extracting objects using 3D reconstruction, feature detection and segmentation (Kawai et al. 2012). In cases in which foreground occluding objects are contextually unimportant, the application has used diminished reality solutions to deal with occlusions Herling and Broll (2010). In such cases, the occluding object is completely removed and the virtual object is shown in its place. Generally, the methods using background information collect and process information on the 3D structures of the scene, which they then take into account. These methods typically achieve good visual quality, but require more processing capacity. Another proposed solution for occlusion management is ‘X-ray vision’, in which the real environment and the augmentation are blended in a manner that creates an illusion of seeing through or inside a real object (Schall et al. 2008). The hypothesis in the mobile work machine context and the project was that AR could be used for projecting information onto the windscreen.

Visibility issues are critical for machine safety, ergonomics and task performance. Therefore, it is important to apply a human factors and ergonomics (HFE) approach. HFE can be defined as: ‘the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance’ (IEA 2000). Czaja and Nair (2006) summarized that ‘the focus in HFE is on studying performance within the context of tasks and environments’. Performance can be evaluated by measuring, for example, time, task completion rate and the number of errors.

Virtual prototyping (VP) was selected for use in this study to support HFE evaluation, as it supports task performance in a real context and provides the possibility to evaluate product features that do not yet exist in real life. A virtual prototype can be defined as a computer simulation of a physical product that can be presented, analysed and tested from various aspects. The process of constructing and testing a virtual prototype is called virtual prototyping (Wang 2002). In recent years, the use of VP has increased in product development processes due to the improved availability and lower costs of VP technologies (Berg and Vance 2016; Choi et al. 2015). Lawson et al. (2016) agreed that the reduction in costs and the enhancement of hardware and software quality have led to VP being widely used in the automotive industry. According to Leino (2015), VP is especially useful in the concept design phase as it enables better integration of the HFE approach into product design and development. Aromaa and Väänänen (2016) agreed that VP is useful in HFE evaluations. Several studies identify VP as a novel prototyping solution for overcoming the shortcomings of conventional prototyping methods. For example, setting up an evaluation experiment using VP in automobile interior design may reduce the expenditure of time and effort compared to setting up physical prototypes (Kim et al. 2011). Kulkarni et al. (2011) agreed that VP uses less time compared to rapid prototyping techniques. In complex assembly tasks, physical prototypes tend to be more time-consuming, costly and prone to errors (Seth et al. 2011). The use of new VP techniques in comparison with traditional systems during design reviews can make interfaces more user-friendly (Kremer 1998). According to Karkee et al. (2011), the use of VP in the design of off-road vehicles reduces the risks associated with extended field experiments, particularly in difficult terrains.

To summarize, visibility is a critical factor in human–machine system design, and see-through technologies offer a potential solution to this challenge. However, as yet, there is insufficient research related to the use of see-through technologies, especially for mobile machines. In addition, to our knowledge, the use of VP in the experimental evaluation of transparency is considered to be a novel design approach. This study will enhance the understanding of operators’ experience and performance when using transparent structures in mobile working machines. In addition, the study shows how to use virtual prototypes in designing novel solutions for industry.

3 Visibility in mobile working machines

For the purposes of this study, four different machine types were investigated in the field to gain an understanding of use contexts and visibility issues when working in machine cabs. A tractor, a harvester, a mine drilling machine and several cargo handling solutions were studied. Field-of-view examples from each of these four machines are shown in Fig. 1. In the case of the first three machine types, the machines were observed while in use, operators were interviewed, photographs were taken and video and audio recordings were made. The fourth, a cargo handling machine, was also visited; however, in this case the main aim was to acquire greater knowledge of different cargo handling solutions.

Fig. 1
figure 1

Field of view from a forest harvester (left), a tractor (upper centre) and a drilling machine (lower centre), and a view of a cargo handling work environment solution (right)

Full size image

The field studies demonstrated that poor visibility can have multiple effects on operator performance and therefore on overall human–machine system performance. Poor visibility can slow the work of the operator due to the need for caution and the time spent trying to get a better view of the operation target, for example, trying to peek behind the harvester head to see the tree. When snow ploughing with a tractor, it can be difficult to see the corner of the plough, which decreases work quality. With underground drilling machines, poor visibility can lead to costly damage such as boom hose breakage due to boom collision. Cargo handling operators need a clear view of the corners of the truck platform when loading containers. In addition, the operator needs to know where the truck driver is. Operator performance may also decrease due to awkward body postures adopted to cope with poor visibility. In addition, operators continuously make decisions when operating the machine based on what they are able to see (e.g. is the tree straight and of the right quality, does the water come from the right hole in the wall, are there pedestrians around).

The main factors that decrease cab visibility are illustrated in Fig. 2. Visibility can be hindered by the cab frame or by machine parts outside the cab (e.g. booms on a drilling machine); weather and light conditions such as fog, rain, darkness or glare; or other objects in the immediate environment, such as machines, trees and buildings. In addition, objects manipulated by the machine can diminish visibility (e.g. a container when using a cargo handling crane, or a haystack when using a tractor). Field studies on these matters are described in greater detail in Kymäläinen et al. (2017).

Fig. 2
figure 2

Factors reducing operator visibility

Full size image

Human factors and ergonomics requirements for visibility were generated from the field studies, literature and standards. The main HFE requirements for the development of see-through technologies were: (1) the operator must be able to trust the information provided by the see-through system, (2) the operator must be able to see the target at all times, (3) the see-through system must support efficient task performance and (4) the see-through system must support the safety of the operator and others in the work area. Requirement 3 was the main initiator of the study presented in this paper.

4 Materials and methods

This section describes the study’s design, participants, data collection and analysis methods, a virtual prototyping system and a test procedure.

4.1 Study design

The main goal was to investigate whether machine boom transparency has an effect on task performance and what level of boom transparency is preferred by the operator. In addition, the placement of overlaid information was investigated using the same approach. Due to the research goal, the experimental approach of Sanders and McCormick (1993) was adopted. In experimental research, variables are manipulated and their effect on human performance or the system is evaluated. A within-subject design was used in the study. The independent variables were transparency level and placement of the overlaid information. The dependent variables were time, distance and experience. In addition to these, participants were given ‘free choice’ to select the transparency that they most preferred.

Three transparency levels were tested: opaque, 50% transparent (T50%) and 80% transparent (T80%) on a scale of T0% (fully opaque) to T100% (fully transparent) (Fig. 3). The two transparencies were selected based on the field studies and discussions of see-through concepts in Kymäläinen et al. (2017) and from a pretest with a developed VE system. During the pretest, no major visual difference was noted between T50% and opaque. This is in line with the results of Harrison et al. (1995), who suggested that performance in the opaque and T50% situations is almost identical.

Fig. 3
figure 3

The three levels of boom transparency used in the study: opaque, 50% transparent and 80% transparent

Full size image

Operator visibility is often diminished when performing drilling. A transparent boom would make it is easy to perform tasks faster and with better accuracy. Therefore, we assumed:

H1.1

Task performance improves when the transparency level of the boom increases.

H1.2

Operators prefer a transparent boom over an opaque boom when performing a drilling task.

The use of augmented reality (AR) technologies is increasing in industry. AR technologies make it possible to overlay task-specific information onto the operators’ field of view. In this study, operator preferences for the location of AR information were investigated by visualizing drilling information for the operators. The drilling information describes the progress of the drilling (see details in Sect. 4.5). In this study, we call the augmented information ‘overlaid information’ because the whole system (machine, environment, information, etc.) is virtual. Two placements of the overlaid information were chosen: windscreen and tunnel wall (Fig. 4). Windscreen was chosen because this placement is often used in industrial AR studies, and tunnel wall was chosen because the focus of the operator is on that area when drilling.

Fig. 4
figure 4

Two placements of overlaid drilling information: windscreen (left) and tunnel wall (right)

Full size image

Machine operators need operational information when performing a drilling task, such as the progress of the drilling (see Sect. 4.5). Placing the drilling information close to the operators’ point of focus (the drilling point) would facilitate drilling. Therefore, we assumed:

H2.1

Machine operators are able to perform their tasks faster and with more accuracy if the drilling information is located near the drilling point on the tunnel wall.

H2.2

Machine operators prefer drilling information to be on the tunnel wall than on the windscreen.

4.2 Participants

The goal was to recruit participants that had previously operated a drilling machine. However, due to project restrictions (e.g. time and ease of access to experts) the participant group was not homogenous. Five participants had previous machine operation experience and five were novices. Nevertheless, during the data analysis the participants were analysed as a group, keeping in mind possible variance due to the different experience levels. This was done because the study focus was on the overall comparison of operators’ task performance and experience between two operating conditions (i.e. two machine boom transparency levels and two overlaid information placements).

Ten participants (nine male and one female) took part in the test (average age 42; age range 26–64). The professions of the participants were: three engineers, three product specialists, a user interface specialist, a programmer, a consultant and a software developer. Two of the participants used virtual technologies frequently, two used them occasionally, five had some previous experience and one had none. Five of the participants had good vision and five wore glasses or contact lenses. Three of the participants wore their glasses under the head-mounted display (HMD) and one took them off because s/he was able to see without them. One participant was colour-blind.

4.3 Data collection and analysis methods

Observation and questionnaires were used as data collection methods. For both independent variables, two questionnaires were provided: the first addressed the operator’s experience of the transparency/overlaid information placement; the second addressed the operator’s overall experience. In addition, the participants were able to select their preferred transparency level (‘free selection’), which was recorded by the observers. The task time and distance of the drill bit from the drilling target were measured with Unity software. The distance was the shortest straight-line distance from the drill bit to the midpoint of the target. Participants’ demographic information was collected with a further questionnaire. To ensure the validity and ethics of the study, the following questions were asked: I felt present in the virtual environment (4 = completely, 3 = fairly, 2 = marginally and 1 = not at all); it was easy to learn to use the joysticks (4 = strongly agree, 3 = agree, 2 = disagree and 1 = strongly disagree); and being in the virtual environment made me feel unwell (3 = strongly, 2 = mildly and 1 = not at all). In addition, participants signed a consent form. All tests were video recorded and photographs were taken. For the data analysis, a repeated measures ANOVA and a paired T test with SPSS were used. In addition, a thematic analysis was adapted for the qualitative analysis (Braun and Clarke 2006).

4.4 Virtual prototype system

This section describes the used virtual prototype system in detail. For systematic preparation of the virtual prototype system, Aromaa’s (2017) approach was adopted (Table 1). The prototype was an underground drilling machine (Sandvik DD421). The system model included a virtual model of the drilling machine and a tunnel environment. In addition, the operator’s hands, drilling information display, actual drilling information and a user interface (UI) for selecting the transparency and the outlines of the booms, cables and cab were provided. Several dynamic model features were created: boom movements, cable physics, hide/show drilling information, hide/show hands, change boom transparency level, change cab transparency, transparencies on/off, change boom outline width and change boom outline colour. The boom was made transparent, but the drill was always opaque (visible). Visual and aural sensory modalities were provided for the operator. Operators received visual feedback during the drilling task (e.g. proximity of bit to target and drilling progress). In addition, drilling and mine environment sounds were provided. Collision detection was not provided in the virtual prototype. Joysticks enabled operators to manipulate the virtual prototype, and their hands were tracked during the free selection phase. Verbal commands could also be given if free selection via the hand-tracking system proved difficult. In such cases, the virtual reality (VR) expert changed the transparency level via a keyboard. The used technologies included Logitech Attack 3 Joysticks, an HTC Vive VR headset, a Leap Motion hand-tracking sensor (attached to the Vive headset), a Unity 5.5, a Blender for 3D modelling, and Inkscape and Gimp for user interface sprites.

Table 1 Test set-up (adapted from Aromaa 2017)
Full size table

4.5 Test procedure

First, the participants were informed of the project topic, a consent form was signed and their demographics were collected. The participants were then given an introduction to the task to be performed with the virtual prototype and instructed how to use the joysticks and operate the machine boom. Each participant received approximately 10 min of training. After this, the participants performed the drilling task, beginning always in opaque mode to gain an understanding of normal operation. Then, the same task was performed with two different levels of boom transparency (T50% and T80%). After that, two different overlaid information placements (windscreen and tunnel wall) (Fig. 4) were used. During these overlaid information placements, the boom was always opaque. In total, participants performed the same task five times, but under different conditions (opaque, T50%, T80%, overlaid information on the windscreen and overlaid information on the tunnel wall). Whenever transparency was being evaluated, the opaque mode was always used first. However, the order of transparent levels was randomized. The order of overlaid information positioning was also randomized.

The participants’ goal was to perform a drilling task (Fig. 5). Participants were required to drill two holes in the tunnel wall of a cave, in a certain order. Drilling was only done with the left boom of the machine. Participants needed to place the drill bit close to the target and start drilling. Information on drilling progress was visualized on the control panel for the participant to see (Fig. 5). When the drill bit was near the target point on the wall, a green circle appeared around a yellow dot on the user interface. The diameter of the circle collider, which detected whether the drill bit was near the target, was 1 m. A blue slide bar indicated when the drill was sufficiently inside the wall and when it was pulled out of the wall. After each task had been performed, the participant completed a questionnaire on the transparency or the overlaid information placement.

Fig. 5
figure 5

A participant performing a drilling task and the view from the head-mounted display. Drilling targets are marked with an X on the wall. Drilling guidance is visible on a control panel: drilling targets on a drilling map and drilling progress bars

Full size image

The participants’ final task involved free selection of the transparency level. The participants were able to select their preferred transparency level using a virtual UI (Fig. 6). They were able to select three parameters: transparency of the boom, outline colour of the boom and thickness of the outline. After performing these steps, the participants answered a final questionnaire regarding their overall experience.

Fig. 6
figure 6

User interface for free selection of transparency

Full size image

5 Results

This section describes the results of the study related to transparency, overlaid information placement and simulation sickness symptoms.

5.1 Level of transparency

5.1.1 Task performance

A repeated measures ANOVA was conducted to compare the effect of transparency level on time (task speed) and distance (drill bit to drilling target). The average time and distance for each level are shown in Fig. 7. There was no significant effect of transparency level on time F(2,18) = 0.217, p = 0.807 (sphericity assumed). In addition, there was no significant effect of transparency level on distance F(2,18) = 1.973, p = 0.168 (sphericity assumed). Therefore, it can be concluded that hypothesis H1.1 is not supported: the transparency level does not have an effect on performance (time and distance). It should be noted that the expert participants took more time to perform the task compared to the novice user group. However, both groups had consistent time performances between situations and, therefore, the results were considered as a single group.

Fig. 7
figure 7

Average time (s) and distance (m) per transparency level (opaque, T50% and T80%)

Full size image

5.1.2 Operators’ preference

A repeated measures ANOVA was conducted to compare the effect of transparency on user experience. Experience was measured with four questions related to preference, performance and visibility (Fig. 8). The results of the question ‘I liked the transparency level of the boom’ are statistically significant F(2,18) = 14.596, p = 0.000 (sphericity assumed). Post hoc tests using Bonferroni correction were further conducted to obtain all possible pairwise comparisons. The analysis shows that there was a statistically significant difference in effect between opaque and T80% (p = 0.001). However, between opaque and T50% (p = 0.083) and between T50 and T80% (p = 0.095), there were no significant differences in effect. There was no significant effect of transparency level on the experience of performance F(2,18) = 3.857, p = 0.040 (sphericity assumed). There was no significant effect of transparency level on the experience of seeing the working target well F(2,18) = 5.037, p = 0.018 (sphericity assumed). There was a significant effect of transparency on the experience of occluding visibility F(2,18) = 22.5, p = 0.000 (sphericity assumed). Post hoc tests using Bonferroni correction were further conducted to obtain all possible pairwise comparisons. The analysis shows that there was a statistically significant difference in effect between the conditions opaque and T80% (p = 0.001) and between T50 and T80% (p = 0.003). Between opaque and T50%, there was no significant difference in effect (p = 0.025).

Fig. 8
figure 8

Experience of opaque, T50% and T80% transparency levels. Mean values (M) and standard deviation (SD) of the participants’ opinions of the transparency levels (5 = strongly agree and 1 = strongly disagree)

Full size image

At the end of the study, the participants were asked which transparency level they preferred most (Fig. 9). Based on the results, seven participants preferred the T80% transparency level (‘I would like to have it as transparent as possible without losing connection with the real world’, engineer, age 37). Three participants selected the T50% transparency level (‘When you saw the boom more, it was easier to point it at the target hole’, UI specialist, age 40). None of the participants selected opaque.

Fig. 9
figure 9

The participants’ preferred transparency level (T50% and T80%) (N = 10)

Full size image

Based on the final questionnaire, all of the participants agreed that transparency could improve their task performance: ‘Due to transparency, I had a better understanding of the big picture and it helped to target the drill’s bit onto the mark on the wall’ (programmer, age 26). All of the participants indicated that they would immediately use the transparency system if it was possible to do so in a real machine. This would make it easier to concentrate on essentials and continuously to see what was happening. In addition, transparency would be employed in tasks where accuracy is needed or where it would help avoid risky situations and collisions. Some participants stated that they would have wanted to see the end part of the boom near to the drill bit. Moreover, they would have liked to be able to see the mechanics of the boom and hoses to detect possible oil leaks or boom damage. Furthermore, they would like to have been able to see the water exiting the drilled hole. They also expressed concern regarding the risk of collision between transparent booms.

At the end of the study, participants were able to select their own preference for the transparency level (‘free selection’). The mean value of their selection was 72.3% for transparency (SD = 20.5, range = 36–100). Seven of the participants selected the outline to be visible (on) and three for it to be invisible (off). From the available colour choices, three participants selected blue, two green, two orange and three no colour.

Hypothesis H1.2 is thus supported: operators like some transparency in a boom. The quantitative and qualitative data analysis methods both showed that the participants preferred a transparent boom (T80%) over an opaque boom. However, the participants did not feel that the transparency level has an effect on task performance. In addition, they said that they were able to see the target area well when working without a transparency system, even though they agreed that the boom occludes visibility.

5.2 Placement of the overlaid information

5.2.1 Task performance

A paired T-test was conducted to compare the effect of information location on performance. The placement of the overlaid information had no significant effect on performance (time and distance) (Table 2). The total time was similar for both scenarios (windscreen and tunnel wall). However, the standard deviation was higher for the tunnel wall scenario. One reason for this could be that one participant took more time to perform the task: after drilling the first hole s/he paused before remembering to drill the second hole. There were no significant differences in distance between the two scenarios (Table 2). As previously mentioned, it is important that the information projected onto the tunnel wall does not cover important real information (e.g. the topography of the wall or the water coming from the hole). The participants also suggested different overlaid information content: distance from the wall, depth of the hole, posture and weight.

Table 2 Mean values (M) and standard deviations (SD) for task performance time (s) and distance (m) with information placement on windscreen and tunnel wall
Full size table

Based on the findings, hypothesis H2.1 is not supported: the location of the drilling information does not have an effect on performance (time and distance).

5.2.2 Operators’ preference

A paired T-test was conducted to compare the effect of information location on participants’ experience. The participants had no clear preference regarding the placement of overlaid information, and there were no significant differences in their experience of the different overlaid information placements (Table 3). However, the participants did seem to have a slight preference for tunnel wall placement, but the differences were small. Regarding the overlaid information, the participants stated that the information should not occlude other important information on the wall. Regarding the placement of the information on the windscreen, the participants would have preferred the information to be in the middle or at the top of the windscreen instead of in the bottom left corner. Some said that the placement should be adjustable by the operator. If the information is on the tunnel wall, the font size should be bigger, the colours could be bolder and greater contrast should be provided. One participant said that detailed information could be shown on the windscreen (e.g. drilling map) and approximate information on the tunnel wall.

Table 3 Mean values (M) and standard deviations (SD) of the experience of the placement of the overlaid information (p < 0.05) (5 = strongly agree and 1 = strongly disagree)
Full size table

Four of the participants preferred placement of the overlaid information on the windscreen (‘If the boom is transparent, the information can be seen on the tunnel wall. If not, the boom can occlude the overlaid information.’ software developer, age 33) (Fig. 10). Five participants preferred the overlaid information to be on the tunnel wall (‘So that when you are already looking at the target, it only needs minimal eye movement to see the information’ product specialist, age 58). One participant was unable to decide between the two placement options: ‘It depends on the situation’ (product specialist, age 37).

Fig. 10
figure 10

Participants’ preferred placement of the overlaid information

Full size image

Hypothesis H2.2 is not supported: the participants did not prefer the tunnel wall over the windscreen. Both placements were equally liked.

5.3 Presence, learning and simulator sickness

Presence, learning and simulator sickness were evaluated in order to consider their possible effects on the results. The participants stated that they felt present during the test. The mean value was 3.5 (SD = 0.53). In addition, the participants agreed that it was fairly easy to learn to use the system: the mean value was 3.3 (SD = 0.48). Nine participants said that they did not experience any strain, headache or other unpleasant symptoms. One participant experienced eye strain due to the boom’s outline (too bright).

6 Discussion

This section discusses the results of the transparency and overlaid information evaluations. In addition, the use of VP in the design of transparency systems is discussed. Finally, the limitations of the research are addressed.

6.1 Transparency

Based on the results, there was no significant difference in performance between the transparency conditions (opaque, T50% or T80%). This is an interesting finding, as it would have been easy to hypothesize that the more transparent the boom is, the better the participants would be able to perform. These performance results might, however, have occurred for a number of reasons. One reason for the actual results could be that operators use multiple senses and information sources when performing a task, and thus, enhanced visibility of the target is only one advantage among many. Another reason might have been the VP test set-up. Participants knew that they were unable to break the machine and therefore they were not afraid of the effects of limited visibility. The test situation also differed slightly from a real situation with regard to boom use, as it was not required for the drill to contact the wall at 90 degrees. Had this been required, the structure of the boom could have blocked the view more. Another consideration is that mining machines have a long development history and their design has already undergone many iterations. Therefore, see-through features may add some value, but the overall performance gain might not be so significant. In this study, the existing machine was retrofitted with the see-through features, which can lead to design conflicts, as the machine was originally made without transparency support in mind. More efficient future versions of mining machines with native see-through support would be able to utilize completely different design approaches.

Lindlbauer et al. (2016) studied the influence of display transparency on task performance in a service clerk scenario. In their study, the purpose of transparency differed from our study. Their purpose was to support both the primary task on the display and the secondary task of detecting background stimuli behind the display. However, it was interesting to note also in their study that the participants’ primary task performance was not influenced by the display transparency. Harrison et al. (1995) investigated transparency in computer interface design. They conducted an experiment in which users focused on icons in the front of the screen, while other images were displayed in the background. Contrary to the findings of our study, their study revealed that transparency level affected response time performance, although no discernible difference in effect was found between transparency levels of 50% and 0% (opaque). It should also be noted that their test set-up differed from ours.

The participants in our study preferred a higher transparency level: they selected the T80% transparency level more often than the T50% level, and in the free selection the mean value was T72%. This suggests that the participants were more comfortable when performing the task with greater transparency. For example, they felt more confident and productive when their visibility of the target increased (e.g. T80% vs. T50%).

Working targets were not the only things that needed to be seen during the tasks. Nine participants did not select total transparency (T100%) because they considered it important to see some parts of the boom in order to be able to detect errors and anomalies: the boom might collide with something, there might be problems with the mechanics, oil can leak from the hoses or the boom might be damaged. Therefore, in addition to being considered as masking components that obscure objects, machine parts need to be viewed as targets requiring the operators’ attention.

In the free selection phase, participants adopted two approaches to using transparency and outlines. The most commonly used approach was that of the boom being clearly transparent and then adding outlines as an aid for understanding the placement of the boom. However, one participant chose a completely opposite approach and used a transparency level of T36% with the boom and no artificial outlines. The free selection phase also shows that there are additional benefits in having the ability to customize see-through features. Many professional software packages have configurable and modular user interfaces, making it possible to specify which parts of the user interface should be present on screen and to change its size, placement and colour scheme, and enabling settings to be saved as a user-specific profile. The same could be applied to mining machines; operators could choose which parts of the boom should be transparent and separately change the level of transparency and outline colour of specific part(s). This customization also applies to the placement of overlaid information. Overlaid information could be moved freely around the windscreen and between the windscreen and tunnel wall projections. Lastly, all of the changes should be accessible during operation to cover different working conditions.

The use of see-through technologies changes the design approach to visibility issues. Earlier, the goal was to decrease the masking effect and blind spots by making structural changes to the machine (e.g. changing the position of a tractor’s exhaust pipe to achieve better visibility (Gilad and Byran 2015)). With see-through technologies the focus is more on providing needed visual cues than diminishing occluding objects. Therefore, with see-through transparency solutions, both the task and the attention of the operator are extremely important to consider: where does the operator need to focus and do they need to divide their attention between different information sources (Harrison et al. 1995). Another design decision that arises is whether all information needs to be visually provided to the operator or could some information be provided via other senses, for example, if a hose is leaking an alarm bell might sound, but no visual feedback would be given. However, information coding and the amount and modality of the information selected need to be carefully considered (Schwarz and Fastenmeier 2017).

6.2 Overlaid information

The placement of overlaid drilling information had no significant effect on task performance (time and distance). It therefore appears that the participants were able to use the information in both placements (windscreen and tunnel wall) in such a way that it did not decrease their task performance. The information provided drilling data and informed the operator when drilling was completed. However, the design of the information presentation could have had an effect on the results. The visualization approach used here was clear and easy to understand. If, however, the information had been more detailed, it is possible that the participants would have been slower when reading it on the windscreen than on the tunnel wall, which was close to the actual working area.

There was no clear preference for the placement of the overlaid information. Five participants selected the tunnel wall, four the windscreen and one both placements. Azuma and Furmanski (2003) studied label placement algorithms and found that users were able to read labels fastest with algorithms that prevented overlap, even if the placement was not ideal. The authors discussed that the objects/labels should not occlude each other and that it is important that overlaid information does not occlude important information in the real environment (e.g. water coming from a drilled hole). What could be most beneficial would be to provide operators with the possibility to manage the placement and size of the information, themselves, while taking into account that it must not occlude any other important issues. The content of the information is also important. The participant who was unable to choose a preferred placement for the overlaid information stated that the placement depends on the task and the content. Therefore, it is important to provide situationally relevant information on correct placement.

6.3 Use of virtual prototyping

This study provides a good example of how to use virtual prototypes when designing future features in human–machine systems. The virtual prototype enabled assessment of the users’ experiences and preferences without having the actual see-through technology available.

The participants expressed no objections to using the virtual prototype features, which indicates that the prototype replicated a real machine well. This supports natural task performance in VE. Therefore, it was possible to adopt an experimental evaluation approach to this topic. However, when evaluating visibility issues, it is important to consider individual differences in eyesight. Some participants used their eyeglasses under the HMD and some took them off because they could see quite well without them. In addition, participants may have experienced discomfort due to limited time for personalizing the headset fitting. For example, there was no inter-pupillary distance (IPD) adjustment between users, and the eye relief in the headset was used in the min–max positions to accommodate glasses. In addition, one subject was colour-blind, which could have had an effect on the selection of outline colours as s/he felt that s/he was not seeing the outline colours as clearly as the other participants were. However, this issue could be addressed by implementing specific modes to support colour blindness (e.g. Hardin 2016). In the case of the VP used in this study, it is possible to make an additional full-screen post-effect shader (transparency and outline achieved in the same way) to modify the colours of the entire scene or to apply a mask to tagged objects in order to change the colour of specific sets of objects.

While the use of VP to test transparency and overlaid information placement proved straightforward, the results were not compared with any physical prototypes or real machines. Therefore, it was not possible to evaluate the validity of the results. However, our comparative study approach provides valuable results regarding the user’s preferences regarding transparency and overlaid information placement. Therefore, the results can be used as a guidance to support next steps in the development of see-through features. In addition, testing see-through features in VP is safer compared to using real-life physical prototypes.

The selection of participants is important when using VP in design because it affects the evaluation of task performance. In this study, the test participants were both novice and expert operators. As mentioned earlier, the expert participants took more time to perform the task compared to the novice operator group. This was because the novice operators paid less attention to correctly aligning the drill bit, whereas the experts, based on real experience, correctly adjusted the bit angle and used more realistic manoeuvring styles aimed at minimizing damage risk and maximizing precision and work quality.

6.4 Limitations of the research

Some limitations may have had an effect on the validity of the research. The training time could have been longer, due to the complexity of the joystick functions. In addition, the group of test participants could have been more homogenous (e.g. the expert participants took more time to perform the tasks). However, the main purpose was to compare operating conditions, not to determine fastest performance times. Furthermore, depth and distance perception in virtual environments does not always correspond to reality. However, in this study, none of the expert participants mentioned this issue.

The drilling task in the virtual environment was not an exact replica of the real task, which could have had an effect on the task performance. There was also an error in the logic of the left joystick: one movement was switched with another. This decreased the natural feel of the operation, especially for the expert operators. In addition, in a real drilling task it is important to drill at a right angle. In this study, all that was required was to position the drill bit near the target, and right-angle positioning was not requested. The participants were informed that the distance would be measured, but the task instructions could have been more precise: some participants may have misunderstood the meaning of the green circle in the UI and may have thought that the drill bit is at its optimal position when the green circle appears, and therefore did not continue to improve the drilling placement. This may have been one reason for the large mean values when measuring the drill bit distance from the target. In addition, the task execution times were short, which could have decreased participants’ engagement with the task.

7 Conclusions

Visibility from a mobile working machine cab is a critical issue regarding task performance, safety and ergonomics. The purpose of this study was to investigate whether the transparency level of the machine boom has an effect on task performance (time and distance). In addition, the effect of overlaid information placement on task performance was evaluated. Operator preferences regarding transparency level and information placement were also studied.

Based on the research, there was no significant difference in performance between the studied boom transparency levels (opaque, T50% or T80%). However, operators preferred the higher transparency level even though it did not affect their performance. It can be concluded that operators are very open to using the transparency feature, but that it does not necessarily improve their task performance.

Placement (on windscreen or tunnel wall) of the overlaid drilling information had no significant effect on operator performance. Both information placements were perceived by the participants to be valuable. This implies that augmented reality overlays should be designed so that the placement of information can be changed by the operators themselves based on their preferences.

The findings of this study can be used when considering novel ways of diminishing occluding objects in industrial machine cab design. In addition, it highlights the importance of holistically taking into consideration all tasks and all systems when designing enhanced visibility. This study also enhances current research knowledge related to the use of virtual prototyping in the evaluation of human factors and ergonomics issues.

This novel study was designed as a preliminary investigation into the performance effects of transparency and overlaid information placement. In the future, the effect of transparency on task performance should be studied further in order to ensure the validity of the findings. Likewise, the nature and the placement of overlaid information are complex topics that require further investigation.