Keywords

1 Introduction

Recent studies on transparent displays have aimed to address new interaction techniques and to identify potential applications such as face-to-face communication systems [1], social engagement tools, augmented-reality messaging applications [2], 3D interaction tools [3, 4]. In particular, a transparent display can be used as a face-to-face collaboration tool between collaborators located on the opposite side, while they use a common user interface projected on the same screen. In such a usage scenario, however, the transparent characteristic of the display potentially generates interruptive (or disruptive) user experience conditions [2, 5], where the objects or environment in background are penetrated to foreground or the background user interface windows are overlapped to the foreground windows. Among these considerations, the overlapped (or layered) window issue can be particularly problematic if a dual-sided transparent display is employed in a collaboration environment, where the two users are situated face-to-face as shown in Fig. 1.

Fig. 1.
figure 1

Transparent display equipment for the experimental environment

Full size image

To address the transparent, overlapped window issue, researchers studied various usage scenarios with a prototype emulating transparent user interface windows with a partial overlap. In [6], the potential use cases of transparent displays are presented with a fabric-based prototype for the collaboration of users on the opposite side. In [7], the usage scenario and usability evaluation of a transparent display prototype that is capable of controlling the transparency level are provided. These publications, however, are focused on evaluating a usage scenario from the application point of view without providing fundamental, empirical experiment results, which can be used to evaluate the feasibility of using a transparent display in a realistic use environment.

To fill such a gap revealed in previous usability researches, we performed a pilot study, which was based on the following question: Can the partially overlapped interface windows with a certain degree of transparency be effectively used in a collaborative work environment using a transparent dual-sided display? We developed an empirical prototype that emulates the characteristics of a transparent display using two conventional displays and associated simulation software. In this pilot study, we conducted the statistical analysis of experimental data in terms of response time and correct response rate to evaluate the user performance in a controlled test environment by programming the transparency level and the size of layered windows.

In an extended research based on the pilot study, we include contrast (or contour) effects in the controlled user interface parameters to further study the feasibility of using highly overlapped windows. This research potentially provides fundamental, empirical user research data such as the maximum allowed layered window size, optimal transparency and contrast level, to the field of user experience research for a transparent display used in a collaboration environment, where two (or more) users share the same display placement on the opposite side.

2 Method

Participants.

30 university students (20 male, 10 female) were paired in 15 groups (the average age of the participants is 27). All participants had normal or corrected-to-normal vision capability. A local committee approved the Institutional Review Board (IRB). Participants gave a user testing consent form.

Equipment.

As shown in Fig. 1, we simulate a dual-sided transparent display by using two conventional 42” LCD displays (resolution: 2560 by 1440 pixels), front-facing embedded cameras and associated simulation software (developed by using the Unity game engine). The LCD displays provide the participants with stimuli, where foreground and simulated background windows with arrow (target and distraction) markers were shown. The behavior of participants was captured by the front-facing cameras and its mirrored image was projected to the participant on the opposite side in real time by using the simulation software. Arrow keyboards were given as an input device.

Stimuli.

Stimuli are provided to participants within a viewing angle of 60° and a distance of 450 mm, as shown in Fig. 1. A foreground window and a background window (4 × 4 bins each) are projected on a simulated transparent display with the variable area of overlap between them: The overlapped (or layered) window size used in this experiment consists of (1 × 1)-bin, (2 × 2)-bin, (3 × 3)-bin cases. Each window presents a unique distractive arrow, and one of the two windows displays a target arrow marker that is distinct from the distractive arrows (among the four arrow types: left, right, up and down). For instance, in Fig. 2(b), a target arrow is highlighted in a red circle, and the other 31 distract arrows are shown. The location of a target arrow is randomly determined: It can be on either the foreground or background window, and on either the overlapped or non-overlapped area. To simulate the transparency condition of the user interface windows, we used four distinct transparency levels defined in an alpha value (a measure in percentage) of a 32 bit-RGBA color space ranging between 0 and 255: 51 (20 %), 102 (40 %), 153 (60 %), and 204 (80 %).

Fig. 2.
figure 2

4 × 3 factorial design: (a) transparency and layered size combination; (b) stimuli based on visual search model.

Full size image

Procedure.

The 4 × 3 factorial design used for our user experiments is illustrated in Fig. 3, where the transparency level factor and the layered window size factor are used. Participants are asked to detect a target arrow marker, which presents along with distractor arrows on the display and to press a matching arrow key on the keyboard provided to them. A total of 192 trials (after 10 practices) are conducted for each pair of participants in approximately 15 min. The position of a target arrow, the transparency level and the layered window size are randomly determined for each trial. The response time (RT) and correct response rate (CRR) are recorded by the simulation software for data post-processing and analysis. Since the study explores the feasibility of collaboration tasks with a transparent display, the experiment session remains active if one of the participants continues to search for the targeted arrow. We conducted user interviews at the end of each task.

Fig. 3.
figure 3

Transparency level control: (a) response time and (b) correct response rate

Full size image

Data Analysis.

Experiment data are analyzed within a pair (collaborator A and B) in terms of RT and CRR. Two sets of data were excluded from the high error rate, and uncorrected data was discarded. Prior to analyzing the performance data, normality tests are performed. Since normality is not assumed in data post-processing, non-parametric statistical analysis techniques are used such as the Friedman test (*Fri.) and Wilcoxon signed-rank test (**Wil.) to assess the differences among transparency levels, layered sizes, and interaction effects for two factors with the significant p-value at a 5 % level.

3 Observation

We analyze the user behavior data obtained from the pilot study in two distinct categories: [Overlapping Case] where the target arrow marker is presented in the overlapped area of the two (foreground and background) windows, and [Non-overlapping Case] where the target maker is in the non-overlapped area. The user performance is analyzed in terms of RT and CRR (averaged values obtained from both the foreground and background users).

Transparent Level Effect.

In Fig. 3, RT and CRR measured at various transparent levels are shown: RT (*Fri.: χ2(3) = 31.062, p < 0.01) and CRR (*Fri.: χ2(3) = 10.113, p < 0.05), where a noticeable variation in both RT and CRR is observed across transparency levels. Figure 3(a) shows that a higher transparency level increases the response time (in both [Overlapping Case] and [Non-overlapping Case]. This fact is potentially because of the visual interference of the background objects in the experiment environment to the user in the foreground in a higher transparency level case.

As shown in Fig. 3(b), the CRR of [Overlapping Case] shows a decreasing trend as the transparency level increases (**Wil.: 20 %/80 %, p < 0.05; 40 %/80 %, p < 0.01), which can be explained by the fact that a lower transparency level (i.e. 20 %) generates a higher contrast factor to the user interface windows, which helps the user correctly detect the target marker. In comparison, CRR of [Non-overlapping Case] measured at the 20 % transparency level is lower than that measured with the 40 % transparency level, which is considered a threshold in the plot.

The 40 % transparency level is the optimal index for the usage scenario of a transparent display with potentially overlapped windows, which is consistent with Harrison’s hypothesis [1].

Layered Size Effect.

Figure 4 shows the user detection performance measured in RT and CRR measured at various overlapped window sizes. In Fig. 4(a), RT values measured in [Overlapping Case] show meaningful differences across various overlapped sizes (*Fri.: χ2(2) = 19.846, p < 0.01), where the (1 × 1)-bin overlapped case results in a faster RT as compared to the (2 × 2)- or (3 × 3)-bin overlapped case, while [Non-overlapping Case] does not show a meaningful variation.

Fig. 4.
figure 4

Layered window size effect: (a) response time and (b) correct response rate

Full size image

As shown in in [Non-overlapping Case] of Fig. 4(b), it is noticeable that CRR of the (2 × 2)-bin overlapped case is higher than that of the other cases. This implies that a smaller visual field (or reduced total area of search) enhances the user detection performance in the non-overlapping region, which makes it relatively easy for the user to find a target marker in the windows. In comparison, the user might have experienced overloaded search areas [6] in the (3 × 3)-bin case, which results in reduced CRR as compared to the (2 × 2)-bin case. Note that the (2 × 2)-bin overlapping case presents a consistently stable RT and CRR performance in both [Overlapping Case] and [Non-overlapping Case]. This experiment result shows that it is promising to use windows with a certain degree of overlapping, not just to pursue as a small area of overlapping as possible.

Transparency and Layered Size Effect Interaction.

Finally, the interaction effects of transparencies and layered sizes were examined. There were significant interaction transparencies and layered sizes in RT performance (Fried, χ2 (11) = 87.568, p < 0.01) and CR (Fried, χ2 (11) = 27.958, p < 0.01). Each transparency level set according to layered size was shown as follows. In the RT performance, all transparency levels revealed significant differences among pairs of 1 and 4bin (Wil, 20 %, p < 0.01; 40 %, p < 0.01; 60 %, p < 0.05; 80 %, p < 0.01), 1 and 9bin (Wil, 20 %, p < 0.01; 40 %, p < 0.01; 60 %, p < 0.05; 80 %, p < 0.01). Both 4 and 9bin showed similar results in RT. Overall, regardless of transparency level, layered size affected the results, as shown in Fig. 5(a).

Fig. 5.
figure 5

Collaborators A and B average: (a) layered size as transparency condition in response time: (b) correct response; (c) transparency condition as layered size in response time; (d) correct response.

Full size image

In the CR performance, of the two pairs of 1 and 9bin (Wil, p < 0.01), 4 and 9bin (Wil, p < 0.01) in the 80 % level showed significant differences, as depicted in Fig. 5(b). Although there were no significant differences of CR, the CR of 1 and 4bin was higher than that of 9bin in the highest transparency level. In addition, each layered size according to transparency level was demonstrated, as shown in Fig. 5(c and d). In the RT performance, the higher the transparency level, the longer the RT, as mentioned in the transparency results section. For CR performance, the CR of the 20 % and 40 % levels was higher than that of the 80 % level. Based on the results, the 80 % level with the 9bin combination may cause the worst performance for users of TDs.

Interface Guideline.

In summary, to guide users, the TD would be most effectively used at a transparency level between 20 % and 40 %. The ‘contrast’ effect may influence all transparency levels during collaboration. In addition, the layered size efficiency showed the best performance in the 1bin condition. Based on the results in the previous section, we suggest that the 40 % level with 1bin is the most effective condition for a collaboration environment. Alternatively, 4bin (25/100 area) presented a consistently stable performance across transparency levels. Therefore, we suggest that 4bin is likewise an efficient condition with broad transparency usage.

4 Extended Study

From the pilot study, we found that appropriate levels of transparency and windows overlapping potentially enhance the visibility of a user interface realized on layered multiple windows: The increase in either transparency level or overlapped window size generally degrades the user detection performance with a certain degree of threshold.

To further investigate the feasibility of using a layered, transparent window condition and to identify a complete set of the interface parameters that need to be tuned for the optimal user performance, we propose an extended user research (work-in-progress), where a depth factor or a contour effect are employed as an additional user interface parameter in the experiment. The use of a depth factor, especially in the overlapped area of the foreground and background windows, may help enhance the user response time to detect the target marker (or generally enhance the user performance). In summary, we employ the controlled interface parameters in our extended study to investigate a complete set of interface parameters for using transparent, layered windows in a work-in-progress:

  • Transparency level

  • Overlapped window size

  • Depth factor

  • Contour effect