Keywords

1 Introduction

Mobile devices are commonly used for the casual creation and manipulation of photos and videos. Actually, they are also increasingly used (semi-)professionally, e.g., by pro-sumers, enthusiast and even experts in film making. In cinematographic production, they often serve as remote control units and help to steer drones or other motorized motion control systems. Mobile devices are particularly attractive as they can combine a remote control unit with the equally necessary display unit (for steering or for reviewing results). They provide multi-touch interaction, general functional versatility, high connectivity, and often an easy set-up on location.

For enthusiast users, an unoccluded view on a transmitted video stream is important, as they create their results with high effort and great care to details. However, so far, the graphical user interface (UI) elements for motion control systems are often displayed as an overlay on top of the video streams, taking up screen space and hence partially occluding the view.

An established way of minimizing the number of visual elements displayed is Progressive Disclosure (PD) [15, 20] which follows the approach of disclosing or “hiding” features into a collapsed or off-screen menu. Items that are less frequently used are then only accessible via a menu. Only when the menu is opened or displayed, the hidden elements become visible again. This minimizes the need for displaying graphical UI elements. For continuous control elements and frequently used elements, however, disclosure might not be a suitable strategy to declutter the screen as they are particularly often needed. Especially when things go wrong, easy access is crucial. Additionally, the perceived affordances [17] of a user interface can diminish when too much is hidden from the user early on. A recently discussed alternative approach and/or extension to PD is Progressive Reduction (PR) [9]. PR follows the idea that interface elements that are frequently used become reduced in terms of their visual appearance. In detail, the characteristics of the used visual variables [4] (e.g. size, form or opacity) can be reduced. As this process happens gradually over time and not abruptly, it exploits the users ability to learn functions, abstract representations and locations of certain interface elements. The adaption of the UI is synchronized to a users usage/learning curve for an application or could also be based on prior knowledge if tracked at system level.

Contribution

In our work, we explored how the combined reduction of size, opacity, icon form and icon visibility of UI elements could help to declutter screens on mobile touch screen devices. We conducted two studies to determine the effects of Progressive Reduction as an adaptation strategy on a camera-based UI for a cinematographic motion control system. In the first study (N = 10), we compared three design alternatives (software joystick, extended software joystick and single knob) to obviate interference due to the UI design. When using touch interaction, usually a diminished sense of precision and control can be observed. Hence, we used the most promising design (extended software joystick) to collect data on perceived control and workload in addition to creativity support and qualitative feedback to compare two reduction strategies (icons-first and background-first) in a second study (N = 18). Our results indicated that it was generally possible to gradually reduce the appearance of UI elements without a major negative impact on the sense of control. However, there also was a certain limit to the amount of reduction that could be applied before the perceived control significantly decreased to an unacceptable level. This could be observed independently from the applied reduction strategy.

2 Related Work

Progressive Disclosure is a well-established concept in HCI literature as, for example, described by Nielsen [15] or Tidwell [20]. In contrast, Progressive Reduction is a rather recently proposed concept and so far was mainly discussed online [9]. Up to now, It has received less attention in the literature, which makes further studying of the subject matter necessary. However, dynamic adaption of UI elements has been investigated before. Most prominently in the domain of augmented reality. In particular, it was used to enhance text readability [8], filter information [13], adapt symbols in camera-based [3] or map-based user interfaces [16]. Adaptation is particulary interesting as it allows to design minimalist user interfaces by exploiting human spatial memory. Spatial memory was also successfully exploited before in UI design and (among other approaches) led to marking menus [14] and even imaginary interfaces [10]. In the case of PR however, only little is known about its side-effects and its limitations.

3 Design Alternatives and Implementation

We implemented three design alternatives (Fig. 1) in a Unity 3D virtual environment on an off-the-shelf Android tablet. As a status quo design we used a software joystick as often found in the wild, e.g., for remote control of drones or in games (Fig. 1a). Here, the left software joystick controls translation and the right software joystick rotation each in two dimensions (left-right, up-down). As the first alternative, we implemented an extended software joystick with a pie menu (Fig. 1b). The pie menu lets users choose, which dimension(s) they want to map onto the software joystick depending on the use case. Additionally, the structure of a pie menu lends itself well to mental and physical learning (e.g., muscle memory) [14]. In consequence, once learned, not all options need to be displayed, which helps to minimize the number of visual elements necessary. As a second alternative, we used a UI that decoupled translational and rotational control to better suit the cinematographic context. In the field, the various axes are often controlled by different operators [12]. The first camera operator often delegates translational moves to a gripFootnote 1. In our case, this could be delegated to an assistance system. In the User Interface, this was presented as a horizontally restricted joystick placed in the centre of the display (Fig. 1c). This single knob could be dragged horizontally with the distance to the centre being mapped to speed of the camera motion. Once content with the settings, the users could “lock” them by swiping upward saving the current direction and speed. For all UIs the button size was at least \(11\times 11\) mm as recommended [1, 2].

Fig. 1.
figure 1

The software joystick (a) and our design alternatives extended software joystick with fully expanded pie menu (b) and the single knob (c)

Full size image

4 User Study Comparing Design Alternatives

To identify the most promising design alternative, we conducted a user study in a controlled laboratory environment.

4.1 Study Tasks

To provide a reasonably varying set of tasks, we asked the participants to execute a product shot, to follow a target moving along a horizontal figure eight (or infinity symbol) trajectory and to roam freely. The product shot is an often-used move, e.g., in advertising. It is rather easy to accomplish as the recorded object is static and only horizontal translation and rotation at low to medium speed is necessary. Following the trajectory of an eight on its side is more complex as the target is constantly in motion. Additionally, translation and rotation in two dimensions is necessary to properly follow it. This approach has already been taken in related work for the evaluation of a semi-automated camera crane [19]. The free roaming mimics a first exploration of possible camera angles and transitions. This task allows users to be more expressive than a task with a pre-defined goal, which is an important aspect in the design and evaluation of creativity support tools [18]. To keep participants interested, we spread multiple animated objects across the scene, thus encouraging exploration.

4.2 Study Design

We used a within-subject design with our design alternatives (3 levels) and study tasks (3 levels) described above as independent variables. To counteract learning effects, we provided each participant with a unique counter-balanced sequence of user interface and task combinations.

4.3 Participants

For the user study, we recruited 10 participants (3 female) with a median age of 26 years. Ages ranged from 20 to 37 years. All participants had normal or corrected-to-normal vision, were acquainted with touch devices and had no prior training in cinematographic camera control. One participant was left-handed. To be able to relate our inferences to the general population, we had no further requirements for the selection of the participants.

4.4 Procedure

We welcomed the participants, informed them about the procedure of the study and handed out a consent form. After having declared consent, we asked the participants to fill out a demographic questionnaire. This was followed by introducing the first design. We asked the participants to carry out the tasks in the given sequence while standing (to mimic an on-set usage context). After completing all tasks with one interface, we handed out a questionnaire asking the participants to rate it in terms of control, workload and creativity support. Then the next UI was introduced and the procedure repeated until all tasks were carried out with all interfaces. Before the debriefing, we conducted a semi-structured interview on their preference and the UIs usability.

4.5 Measurements

To collect quantified data on workload, we used the Task-Load Index (TLX) from NASA [11]. For the ratings on creativity support, we use a limited version of the Creativity Support Index (CSI) by Cherry and Latulipe [5]. We did not ask on all dimensions as some were, by design, not supported throughout all conditions, such as ‘collaboration’. Thus, we only included questions on dimensions that were featured by our user interfaces, in particular exploration, motivation and enjoyment. For data on perceived control we handed out a modified version of the sense of control scale (SCS) provided by Dong et al. [6]. As the CSI is based on the TLX, they both use the same 20-point rating scale for each item. To be consistent with the prior items and to minimize confusion for the participants, we also used this report format in our version of the SCS.

4.6 Data Analysis and Results

For the data analysis, we used non-parametric tests (Friedman’s ANOVA, Wilcoxon Signed-Rank) to test for statistical significance. A Bonferroni correction was used for the post-hoc tests to compensate for pairwise comparisons (\(\alpha ^{*}=.016\)). Post-hoc tests were only conducted after a significant main effect was found.

Fig. 2.
figure 2

Summary of the collected data of our study on design alternatives (a) and progressive reduction strategies (b)

Full size image

We found no significant difference for workload (\(\chi ^{2}(2)=2.0\), p \(\le \) .368) and creativity support (\(\chi ^{2}(2)=3.128\), p \(\le \) .209). However, for control we found a significant main effect (\(\chi ^{2}(2)=8.359\), p \(\le \) .015). Post-hoc pairwise comparison between the extended software joystick (Mdn = 82.5) and the single knob (Mdn = 45.0) indicated the extended software joystick could outperform the single knob (Z \(\le \) 2.601, p \(\le \) .009, \(\eta ^{2}\le .677\)). Comparing the extended software joystick to the traditional joystick (Mdn = 67.5), we could not find a significant difference (Z \(\le \) 2.310, p \(\le \).021, \(\eta ^{2}\le .534)\). Comparing the traditional joystick to the single also showed no significance (Z \(\le \) 1.543, p \(\le \) .123, \(\eta ^{2}\le .238)\).

In conclusion, the extended software joystick seemed most promising to us as it reached the highest absolute score in the control rating (Mdn = 82.5), could outperform the single knob (p \(\le \) .015) in terms of control (Fig. 2a) and as it was also preferred when asked about in the interviews. We therefore used it to implement and evaluate two reduction strategies.

5 Reduction Strategies

Visual variables can be reduced in various ways. In our designs, we reduced the appearance of user interface element in terms of size, opacity, form and visibility.

Fig. 3.
figure 3

Our reduction strategies icons-first (a) and background-first (b)

Full size image

To estimate the effects of the gradual reduction, we compared two strategies: icons-first (IF) and background-first (BF). To make the transition between the levels of reduction less abrupt and to provide easily identifiable icons even in small sizes, we introduced an intermediate reduction state for both strategies (Fig. 3). In this intermediate state the icons transition from their original appearance (e.g., “cross with arrows”) to a simplified version (e.g. “plus”) when the overall opacity is reduced to a value below 75%. As the next step, they disappear in different ways depending on the strategy. With the icons-first strategy, icons disappear before the background disappears (opacity level below 35%, Fig. 3a). In contrast, with background-first the background disappears before the icons (opacity level below 35%, Fig. 3b). To counteract a diminished performance due to Fitts’ Law [7] we kept the actual touch-sensitive areas of the UI elements to their original size of \(11\times 11\) mm.

6 User Study Comparing Reduction Strategies

To estimate the effects of progressive reduction in general and of the icons-first and background-first strategies in particular, we conducted a second user study in a controlled laboratory environment.

6.1 Study Tasks

To isolate the effect of the reduction strategies in this study, we used a roaming task similar to the previous study. We wanted to ensure that the participants would use every function an equal amount of the time. Hence, we altered the task from a free roaming task to a prompted roaming task. Here, three boxes were displayed on top of the screen prompting which settings to chose and which camera motion to carry out. The left box would indicate the setting for the left joystick. The right box prompted the setting for the right joystick and the center box the camera motion to be executed. All three boxes changed prompts every 10 s. The sequence was chosen in a pseudo random fashion to avoid unreasonable combinations such as moving up with one joystick and down with the other simultaneously.

6.2 Study Design

To avoid interference due to a learning effect we chose a between-groups design. Each participant therefore only experienced one strategy.

6.3 Participants

We recruited a total of 18 participants (7 female) with ages ranging from 19 to 33 years and a median age of 22.5 years. All had normal or corrected-to-normal vision, were acquainted with touch devices and had no prior training in cinematographic camera control. Two participants were left-handed. Similar to the previous study, we had no further requirements regarding the sample.

6.4 Procedure

As in the previous study we first welcomed the participants, asked for consent and collected demographic data. Then the participants were subjected to a user interface with a given reduction strategy and the above-mentioned prompted roaming task. Again, the participants were asked to perform the tasks while standing. After the first 5 min the PR of the user interface began. The interface was reduced further every 5 min until the final reduction step (invisible or imaginary UI [10]) was reached. At the final stage the participants performed the prompted tasks again for 5 min. Having performed a study task at a given reduction level for 5 min, we asked the participants to rate their sense of control. Data on workload and creativity support was only collected at given points of interest (after 5, 25 and 35 min) to keep interruptions to a necessary minimum. In total 35 min were necessary to perform the tasks with all levels of reduction. Having completed the tasks, we debriefed the participants.

6.5 Measurements

To rate the perceived sense of control, workload and creativity support, we applied the adapted questionnaires from our previous study.

6.6 Data Analysis and Results

Similar to the preceding study, we used non-parametric tests (Friedman’s ANOVA, Wilcoxon Signed-Rank) to test for statistical significance. Also, a Bonferroni correction was applied to the post-hoc tests in order to compensate for pairwise comparisons (\(\alpha ^{*}=.016)\). Post-hoc tests were only conducted after a significant main effect was found.

Overall, we found no significant difference between the reduction strategies for control (Mdn\(_{IF}=75.0\), Mdn\(_{BF}=65.0\), Z \(\le \) .00, p \(\le \) 1.000), workload (Mdn\(_{IF}= 49.16\), Mdn\(_{BF}=49.16\), Z \(\le \) .036, p \(\le \) .971) and creativity support (Mdn\(_{IF}=66.0\), Mdn\(_{BF}=55.0\), Z \(\le \) .361, p \(\le \) .718).

Regarding the data plot (Fig. 2b) we conducted further significance tests comparing the data gathered after 5, 25 and 35 min for each dimension. For workload we found a significant main effect (\(\chi ^{2}(2)=20.111\), p \(\le \) .001). Post-hoc pairwise comparison between 35 and 5 min (Z \(\le \) 3.376, p \(\le \) .001, \(\eta ^{2}\le .633)\) as well as 35 and 25 min (Z \(\le \) 3.201, p \(\le \) .001, \(\eta ^{2}\le .569)\) indicated significant differences. No effect was found comparing 25 to 5 min (Z \(\le \) 2.289, p \(\le \) .022).

For creativity support we found a main effect (\(\chi ^{2}(2)=13.914\), p \(\le \) .001). Also, post-hoc pairwise comparison between 35 and 5 min (Z \(\le \) 3.028, p \(\le \) .002, \(\eta ^{2}\le .572)\) as well as 35 and 25 min (Z \(\le \) 2.534, p \(\le \) .011, \(\eta ^{2}\le .357)\) indicated significant differences. Also, no effect could be found comparing 25 to 5 min (Z \(\le \) 2.226, p \(\le \) .026).

For control we also found a significant main effect (\(\chi ^{2}(2)=30.629\), p \(\le \) .001). Post-hoc pairwise comparison showed a significant difference between all measurements with 35 and 5 min (Z \(\le \) 3.741, p \(\le \) .001, \(\eta ^{2}\le .778)\), 35 and 25 min (Z \(\le \) 3.730, p \(\le \) .001, \(\eta ^{2}\le .773)\) and 25 to 5 min (Z \(\le \) 2.773, p \(\le \) .006, \(\eta ^{2}\le .427)\).

7 Conclusion

We explored how the combined reduction of size, opacity, icon form and icon visibility of user interface elements affected the perceived control, workload and creativity support of a camera-based user interface. In a first study (N = 10), we compared three design alternatives (software-joystick, extended software-joystick and single knob). We used the most promising design (extended software-joystick) to compare two reduction strategies (icons-first and background-first) in a second study (N = 18). Our results indicated that it was possible to gradually reduce the appearance of UI elements without a major negative impact up to a certain amount of reduction, after which it became unacceptable. This was observed independently from the applied reduction strategy.