We used an iterative approach including feedback from persons with VI to design and develop several early prototypes of the two assistive functions and the different information representations. In the following, we report a formal evaluation of the current prototype.
Participants. Participants were reimbursed with 12 EURO/hour. Three individuals with VI participated in the study (P1: 20-year-old male, severe binocular vision loss, 0.04 visus; P2: 24-year-old female, binocular low vision with good color vision, 0.14 visus; P3: 81-year-old female, central vision loss, 0.17 visus). P1 and P2 tested both assistive functions. They had no prior experience with AR devices. P3 tested the clock augmentation and provided feedback on an earlier version of the emotion representations; she has had extensive experience with the HoloLens due prior testing of other research prototypes at our institution. Unfortunately, data collection was interrupted by the Covid-19 pandemic. Although, the generalizability of the current dataset may be limited, we are convinced that it provides constructive insights for future empirical studies and interface design.
Apparatus. The AR system is implemented in Unity 3D with the Mixed Reality Toolkit from Microsoft and deployed to a HoloLens. A computer—connected via a dedicated WLAN Router—using the virtual input functionality from the Microsoft HoloLens App controls the application. We implemented the app on the wristwatch in Java (Watch OS 2/Android). We use the touch screen of the devices as input.
Study Design. Our primary aim was to test different information representations. Thus, we did not use the full systems described in the above “Prototype” section. This simplified the study setup and removed potential confound variables. In each study, we only display the augmentation, i.e., show the visualization for an emotion without a corresponding conversation and a time of day without the triggering gesture.
Both studies consist of four conditions, presented in randomized order for each participant to mitigate order and training effects. In the emotions study, one of four different representations was presented in each condition, namely: one textual (alphanumeric) representation, henceforth termed “Text”, and the three symbolic representations Emoticons, Abstract, and Colors (see Fig. 2). In each condition, we displayed representations of the three emotions happy, sad, and angry (following
[14]). Each condition consisted of 9–15 training and 15 test trials. Maximum font size of the text was chosen such that the longest word (in German) fit in the field of view of the HMD (approx. visual angle of 17\(^\circ \) vertical and 34\(^\circ \) horizontal). In the Emoticons condition, we used default emoticons from IOS, as had been suggested by P3 in a prior iteration of the design process. In the Abstract condition, we simplified the emoticons by only displaying the mouth, and for ‘angry’ the eyebrows. In the Colors condition, we asked each participant to select their own preferred three colors (from a set of seven), one for each emotion, for the reason that there is no wide-spread consensus on the association between colors and emotions.
In the clock study, we presented the time either directly in the HoloLens (without augmenting a physical clock or triggering gesture) or on a wristwatch, either using a symbolic (analogue clock face) or numerical (digital time) representation (see Fig. 3), resulting in four conditions each consisting of four training and six test trials. Here, the wristwatch conditions serve as baselines.
We presented the stimuli in the HoloLens on the physical accommodation plane of the device (2 m distance) and a height of 50 cm resulting in 14.2\(^\circ \) of the visual field and locked to the center of the screen (therefore no calibration of the HoloLens was required). The wristwatch Huawei Watch 2 has a round display with a screen size of 3.05 cm, resulting in 17.3\(^\circ \) of the visual field at 10 cm distance or 5.8\(^\circ \) at 30 cm distance. Each stimulus was displayed until the participant gave an answer regarding the perceived time or emotion.
Procedure. We offered to meet with participants at the nearest station of public transport. When arriving in our lab they sat down on a chair and were introduced to the HoloLens. After filling out a consent form, we reminded the participants that they were allowed to abort the study at any point or take breaks in case they felt uncomfortable.
Clock. We first confirmed that the participants could perceive the clock. Participants were allowed to freely chose the distance between their eyes and the wristwatch. Afterwards, we started the study by showing the first training trial. The participant was asked to read the time aloud as soon as they were able to recognize it. They were allowed to read the time in any format they wish (e.g., “half past 2”, “two-thirty”, etc.). We recorded the accuracy of participants’ answers and their response time.
Emotional Expressions. First, we asked each participant to choose a mapping between the three emotions and three of the seven colors. Afterwards, we displayed all visualizations and confirmed that our assumed mapping for Emoticons and Abstract aligned with the expectations of the participant. Next, we displayed the stimuli analogue to the clock study and the participant had to name the perceived emotion. A button press displayed the next stimulus and response times and accuracy were recorded.