Taking the results from the qualitative study into account, the user interface was further improved and evaluated by conducting an online survey. A special focus was placed on the presentation format for the current traffic situation and the design of the warning instructions.
5.1 Study Design
The first part of the study dealt with the iconography. Participants had to both identify visualized road users and evaluate the different icons for a road user type in direct comparison. Therefore, three revised icons for each type that showed insufficient detection rates during the preliminary study were presented to identify the optimal representation with regard to identifiability on a smartphone screen. There was also a focus on the distinguishability of road user pairs that had often been confused: pedestrians vs. wheelchairs and bicycles vs. motorcycles.
The second part of the study broached the issue of preferred illustration for safety critical situations. Five different animated sequences were laid out to the participants, all in common mobile device size.
The first sequence shows a colored (red) geometrical figure (circle) with a growing opacity level within the collision warning phase (see a section of it in Fig. 4):
The second sequence shows also a colored geometrical figure (red circle) with a growing opacity level within the collision phase, but the size of the circle form increases as well (see a section of it Fig. 5):
The third sequence displays an increasing of the icons’ size within the collision phase, without any coloring (see a section of it in Fig. 6):
The fourth sequence displays a three-tiered coloring approach (green – yellow – red), in a safe phase, short before a critical situation and during collision time (see sections of it in Fig. 7).
The fifth sequence shows a combination of coloring the icon and the size of the icon within the collision phase (see a section of it in Fig. 8).
Similar to the preliminary study, all sequences were evaluated regarding preferences, information completeness, deflection, and situation assessment. Last, demographic data, mobility behavior and technical self-efficacy [19] as user characteristics were surveyed.
6-point Likert-scales were used for all rating tasks (min = 0 “no agreement at all”, max = 5 “full agreement”). The level of significance was set to α = .05. Parametric statistical methods were used to analyze the data and crosschecked by their non-parametric counterparts if there were slight violations of requirements. However, for clarity and legibility reasons only the results from parametric procedures will be reported.
5.2 Sample
186 (N) participants replied to the questionnaire. Altogether, 53.2% were male and 46.8% female. The mean age was 37.7 years (SD = 15.6), ranging from 16 to 86 years. The educational level of the sample was rather high: 66.1% of participants had a university degree, 19.9% graduated from high school and 8.6% completed vocational trainings. Furthermore, the sample showed a high average technical self-efficacy with M = 3.54 (scale maximum = 5, SD = 1.08).
The vast majority of participants owned a smartphone (92.5%, n = 171). Map and navigation applications were used frequently by 28.1% and occasional by 46.8% of those. Of all participants, 93.5% owned a driving license. However, 86.2% mentioned that they often cover journeys on foot and 9.9% still from time to time. To better differentiate between preferred modes of transport, groups for pedestrians, bicyclists and wheelchair user were classified depending on usage frequencies:
All participants stating a rare use of bicycles (never to max. one time a month), but a high frequency of walking were defined as pedestrians (44.5%, N = 82). Further, participants stating a rather regular use of bicycles (several times a month to daily) were defined as bicyclists (50.0%, N = 93). The third group was identified due to their use of a wheelchair (4.8%, N = 9).
5.3 Results: Icon Design
Taking the icons Wheelchair, Motorcycle, Truck and Tram into account (not identified correctly according to the ISO mark in the preliminary study), two to three revised icons were presented and in comparison to another evaluated (see Tables 3 and 4).
Table 3. Frequency of chosen traffic icon for wheelchair user and motorcycle (N = 186).
Table 4. Frequency of chosen traffic icon for truck and tram (N = 186).
The most selected traffic icon for Wheelchair is Icon 3 with a total of 68.3% (N = 186, see Table 3). The icon shows detached individual parts of a wheelchair user from a top view. Still, some participants expressed their wish to visualize the Wheelchair like the “common 2D symbol from side view”.
An expected confusion with other traffic participants (e.g. pedestrians) could be avoided in all three visual representations: In several identification tasks, in which the participants had to decide whether the presented icon is a pedestrian or a wheelchair user, the traffic icon was successfully identified from at least a total of 87.6% (N = 186).
Further, the most selected traffic icon for Motorcycle is Icon 2 with 50.0%, (N = 186), followed by Icon 3 with 41.9% (see Table 3). Here, a closer look to the identification task, in which the participants had to decide whether the presented icon is a motorcycle or a bicycle shows, that both icons were identified successfully (Icon 2: 80.1% and Icon 3: 82.3%). Here, Icon 1 was not identified correctly: 46.8% believed it to be a bicycle and 44.1% identified it as a motorcycle. The icons vary from one another with different helmet sizes of the motorcycle driver and thickness of wheels.
Regarding the traffic icon for Truck, 75.8% selected Icon 2 as preferred representation (see Table 4). The Icon 2 shows a vehicle from top view with a detached driver cabin. With 57.0%, the representation most selected for Tram was Icon 3, followed by Icon 2 with 40.3% (see Table 4). The preferred icon shows a clear distinction between two railway wagons and a pantograph on one of the wagons. Also the connection to further wagons is portrait, again from top view.
5.4 Results: Feedback Design
The presentation of the results of the feedback design will be structured as follows: First, the comparisons between the possible sequences will be presented. Second, user group specific differences will be analyzed.
Overall Comparison of Sequences.
As can be seen from both Figs. 9 and 10 several differences in the evaluation of the different possible visualizations were found.
First, the participants were invited to evaluate, whether the information provided was sufficient. Overall, the sequences evaluation differentiated significantly from each other F(4,732) = 66.76, p < .001). A pairwise comparison shows that only sequence A and E did not differentiate significantly (p ≥ .05).
A closer look shows that sequence B has highest approval rate (M = 3.56, SD = 0.97), followed by sequence A. The evaluation of sequence C reveals a slight disapproval to the given information (M = 2.17, SD = 1.36).
Next, the participants were questioned whether the sequence contained too many distracting elements, which was rejected (see Fig. 9 (middle): all M < 2.5, whereas 5 = max. agreement). Overall, the sequences evaluation differentiate significantly from each other (F(4,736) = 20.95, p < .001). A pairwise comparison shows that only sequence A and B did not differentiate significantly (p ≥ .05). The lowest approval rate scored sequence A, the colored geometrical figure with a growing opacity level (M = 1.55, SD = 1.01).
Third, the assessment of the situation was evaluated. Here, four out of five sequences were agreed upon an immediate assessment of the situation (see Fig. 9). Again, almost all sequences are evaluated with a significant difference (F(4,720) = 56.98, p < .001). Due to a pairwise comparison sequences D and E do not differentiate significantly (p = .146). Sequence B (color and size changing geometrical figure) has the highest approval rate (M = 3.64, SD = 0.99), whereas sequence C scores the lowest agreement rate (M = 2.26, SD = 1.30).
After that, the participants were asked to decide whether they needed further help to interpret the situation displayed correctly (see Fig. 10), which was rejected (see Fig. 10 (left): all M < 2.5, with 5 = max. agreement) although the sequences show significant differences in their rating (F(4,720) = 30.93, p < .001). The pairwise comparison shows that the evaluation of sequence C differentiates from all other sequences significantly (p ≤ .001). The increasing icon size (sequence C) shows the highest agreement on further help (M = 2.36, SD = 1.46), whereas sequence B shows the lowest approval (M = 1.41, SD = 1.15).
An increasing road safety by using the application was evaluated next. Here, the overall approval was rather low (see Fig. 10). Only the sequences A (M = 2.52, SD = 1.51) and B (M = 2.73, SD = 1.56) could score an average approval over 2.5. Again, all sequences show significant differences in their rating (F(4,732) = 43.64, p < .001). A pairwise comparison shows that only sequence D and E did not differentiate significantly (p ≥ .05). Sequence C scored the lowest approval.
Finally, the participants were asked, if the sequence shown was perceived well. To sum up, the sequences were evaluated significantly different from one another (F(4, 724) = 57.18, p < .001), except sequences D (three-tiered icon coloring) and E (three-tiered icon coloring and size increasing) as a pairwise comparison showed (p ≥ .05). Sequence C scored lowest approval (M = 1.69, SD = 1.30), whereas sequence B had the highest approval rate (M = 3.20, SD = 1.29).
Summarizing the iterative evaluation, sequence B showed the highest approval rates considering all statement agreements and given qualitative feedback. A closer look at the sequence reveals a warning icon sign combined with a text at the top of the screen (see a section of it in Fig. 11), faded in, when the situation is becoming increasingly serious. All sequences share this information.
Further, a red colored geometrical figure overlaid on the traffic icon of the upcoming vehicle is shown. The geometrical figure, a circle, increases in size and opacity level by coming closer to the own position, here, displayed as purple arrow. The combination of figure size, opacity level and warning reference is the recommended visualization for a collision warning in our smartphone application.
User Diverse Comparison of Sequences.
To gain a first insight of possible user diverse evaluation patterns, further statistical analyses were performed, addressing different mobility groups, gender and age.
First, a comparison of the sequence evaluation by the mobility groups (bicyclist, pedestrian and wheelchair user) was conducted. No significant differences of the mean agreement ratings could be identified, revealing a joint result of the sequence evaluation.
Further, gender was focused as possible influencing user factor. Again, no significant difference in the evaluation could be found.
Nevertheless, concerning age several indications could be identified. Whereas no significant difference could be found regarding sequence A, sequence B showed a negative, significant correlation (r = −.166, p = .023, N = 186). Younger participants agreed significantly stronger to the statement, that the sequence provided sufficient information. Another finding indicates, that older participants agreed significantly stronger to the statement regarding help to interpret the situation correctly (r = .192, p = .009, N = 186).
Sequence C as well as sequence E showed no significant differences according to age. The only further finding regards sequence D: here, older participants agreed significantly stronger to the statement that the road safety will increase, if they use the application (r = .184, p = .012, N = 185).