Introduction

Uncertainty is a lack of information [1] and therefore uncertainty is ubiquitous in everyone’s life. In many domains where data and information are fraught with uncertainty (e.g. physics, meteorology or the method of data collection itself [2]), temporal information plays an important role [3]. Based on empirical values, we all become familiar with the temporal uncertainty of a train ride or planning a holiday trip [4]. Especially for train trips, there are many statistics that demonstrate the probabilities and the typical extent of temporal uncertainty. In October 2015, for example, approximately one-third of the inter-city trains in Germany were delayed a minimum of five minutes [5]. Therefore, a planned holiday trip by train can be delayed already at the start. Similarly, other means of transport may be delayed, e.g. due to a delayed outbound flight or traffic jams, and after further potential issues during the trip, the end becomes increasingly difficult to predict.

A temporal activity generally consists of three components, which can be described as ‘Start’ (\(\mathcal {S}\)), ‘Duration’ (\(\mathcal {D}\)) and ‘End’ (\(\mathcal {E}\)). For a certain activity, these three components are in the relation \(\mathcal {S}\) + \(\mathcal {D}\) = \(\mathcal {E}\) to each other. Hence, each component is defined by the other two parts. Typically, each of these three components can be uncertain, which significantly complicates the relationship between \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\).

Scheduling is the planning of times, at which particular activities will happen [6] and it usually refers to future activities – in general as well as in personal use. For each of the three components of an activity (\(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\)) we distinguish between variable (e.g. ‘in between 12:30 h and 13:30 h’) and fixed (e.g. ‘either at 12:30 h or at 13:30 h’) occurrences. A variable activity could be a sightseeing trip during the holiday, which lasts between four and five days, whereas a fixed activity could be the flight itself, which starts either at 12:30 h or at 13:30 h. In addition, a probability distribution for the uncertain components can be specified. If a probability distribution is known, it can be either cumulative (e.g. ‘the activity is expected to end at 13:15 h, but may last until 13:30 h’) or discrete (e.g. ‘the activity ends at 12:30 h with 70% or at 13:30 h with 30% probability’).

Users may find it difficult to quantify temporal uncertainty in an activity [7,8,9]. Drawing these activities that may also have a probability distribution, requires users to externalize their internal knowledge. One method to help give this input in a faster and more reasonable way is using an interactive visual representation of the task [2]. Visualizing uncertainty, in general, has been evaluated in numerous studies [3, 10,11,12]. There are several visualization approaches for temporal uncertainty as well [13, 14]. Microsoft Project is one commercial solution to visualize uncertainty in a schedule using options to quantify activities as optimistic or pessimistic as well as PERT-like network diagrams [15]. While these approaches cover several different aspects of uncertainty visualization, none of them offer a complete solution for drawing, visualizing and editing schedules containing activities with temporal uncertainty to externalize implicit knowledge.

With our new approach, we provide the following contributions:

  • An extension of existing methods for visualizing temporal uncertainty that enables the user to display

    • certain and uncertain activities

    • fixed and variable components of activities

    • indefinite activities

    • probability distributions for uncertain components.

  • A design-centred development of visualization and application for interactive planning with temporal uncertainty.

  • A sketch-based interface to enter schedules that include both certain and uncertain activities with all the above characteristics.

  • A quantitative user study to evaluate both the system usability and the user performance. Therefore, the study was split into two tasks: a drawing assignment and a reading assignment.

Related Work

The visualization of time-dependent data and schedules is a well researched problem.

Charts and Diagrams

Gantt charts are one of the most frequently used visualization techniques used for planning activities [16]. Each activity is depicted by a bar. Its leftmost position on a time axis indicates the start, whereas the width represents the duration of an activity. The activity description is usually displayed as textual labels in the left part of the diagram or within their respective activities. The main advantage of Gantt charts is their simplicity and similarity to bar charts, which are intuitive and self-explanatory [17, 18]. However, Gantt charts are not suitable to visualize activities containing temporal uncertainty, since every bar has a fixed position, i.e. a fixed start and a fixed duration [13]. To compensate for this disadvantage, several approaches have been developed that form a good basis for our new research.

Program Evaluation and Review Technique (PERT) diagrams were developed in 1958 [19] and are used for scheduling tasks since then [20, 21]. In a PERT diagram, an activity is represented as a table with the following properties:

  • earliest starting time (EST) and earliest finishing time (EFT)

  • latest starting time (LST) and latest finishing time (LFT)

  • a (minimal) duration and a buffer time

The buffer time describes the difference between the minimal and the maximal duration of the activity. With these characteristics, a PERT diagram is able to visualize both certain activities (EST = LST and EFT = LFT \(\rightarrow\) buffer time = 0) and activities with temporal uncertainty (EST \(\le\) LST and EFT \(\le\) LFT \(\rightarrow\) buffer time > 0). A complete schedule consists of several individual PERT diagrams in which constraints can be visualized as arrows, similar to Gantt charts. Even though the PERT diagrams can be organized in chronological order, it is not trivial to determine the total time span of a schedule [22]. Besides this problem, the obvious disadvantage is that the properties are represented as text and not in visual form [13].

With PlanningLines [13] Aigner et al. introduced a new technique in 2005 that combines the advantages of Gantt charts and PERT charts. PlanningLines are designed as bars, similar to Gantt charts, and accordingly the scalability for large project plans has been retained. They also allow the visualization of uncertainty with the same properties as in PERT diagrams. The start interval between EST and LST is visualized as an open bracket and the end interval between EFT and LFT as a closing bracket. The (minimal) duration of the activity is visualized as a dark bar within these brackets. The optional buffer time is represented as a lighter-shaded extension of the respective bar, divided equally between both ends.

Drawing

Greis et al. [2] developed a set of various sliders to quantify uncertainty for an interactive input. These sliders provide different configurations like Fixed Range Slider or Advanced Flexible Range Best Estimate Slider to specify uncertainties with several degrees of freedom. This set of sliders has been adoped by Kleemann and Ziegler [23] and is also a starting point for our design. A sketch-based input has been already used in scientific fields like mathematics [24, 25] for more than 15 years, as most visualisations and peripherals are based on a 2D interface [26]. Furthermore, sketch-based interfaces are especially suitable for application areas with predominantly beginners and inexperienced users [26, 27].

Lee et al. presented a system to support users during sketching [28] with dynamic shadows, to give them an idea of a possible outcome. This technique is still used in modern research [29, 30].

Uncertainty Visualization

Several studies were conducted to visualize (temporal) uncertainty [2, 13, 26, 28, 31]. MacEachren et al. evaluated eleven techniques (e.g. location, orientation or fuzziness) and a number of icons to visualize temporal and spatial uncertainty. As a main result, they propose that icons are not suitable to visualize any kind of uncertainty. Pictorial representations such as icons demand an understanding of the underlying uncertainty from the viewer, and the viewer also needs to be capable of interpreting the metaphor of the icon correctly [3]. Instead, MacEchrean et al. recommend the use of uncertainty visualisation techniques that add a small error to the data but allow a faster assessment of the uncertainty presented.

Gschwandtner et al. conducted a survey to evaluate visualizations for temporal uncertainty [14]. The goal of this survey was to evaluate which visualisation should be used for quantitative uncertainty (with knowledge of the probability distribution) and qualitative uncertainty (without such knowledge). For quantitative uncertainty, the survey indicates that a gradient brush performs best, although it is not the most popular visualisation among the participants. For qualitative uncertainty, ambiguation performs best and is also the favourite visualization among the participants. Since Aigner et al. uses ambiguation to visualize the buffer time in their PlanningLines approach and the results are also used in recent publications [32, 33], we decided to use this technique as well for the representation of the uncertain parts of our visualisation.

Uncertainty in data is often visualized with error bars. However, error bars are often misinterpreted by users [31, 34]. Instead of evoking a sense of uncertainty, they are frequently interpreted as high certainty in the data [35]. Value labels inside the bars are usually assumed to be more certain than value labels outside [36]. Brackets instead of error bars can help to facilitate the correct interpretation. Aigner et al. use similar brackets in their PlanningLines approach [13] to visualize the time span of an activity.

Design Process

Fig. 1
figure 1

A schematic visualization of our design process, inspired by storyline approaches [37, 38]. The figure shows three stages of the development of our visualization and application—research, the design process itself and the evaluation phase. To plan ideas and express different possibilities, the designs were sketched on a whiteboard [39]. The design process is divided into three parts to show the dependencies between the visual design, the intended interaction and the resulting application. The interaction for drawing constraints (A) is quite similar to the input of Tubes with a stylus and therefore, no additional snapshot is included

Full size image

The iterative design process for our new visualization was structured in three stages: (1) research, including participartory meetings, (2) the design of the visualization and the interaction with the corresponding application, and (3) a quantitative user study to evaluate our approach. Figure 1 shows an overview of this design process.

Research Phase

The research phase was divided into two tracks. On the one hand, there was a requirements assessment process, in which potential end users were asked about their design ideas for a visualization of uncertainty, their needs for an application supporting scheduling with temporal uncertainty, and their suggestions or interactive features of the prototype. On the other hand, a traditional paper research was performed. In this process various uncertainty visualizations like the metaphor of elastic bands, springs or paint strips [10] or even PlanningLines [13] were investigated. Furthermore, visual marks to encode uncertainty [3, 14, 31] were researched. To interact with the software, different approaches for input of uncertainty like sliders [2], sketching [26, 28, 40] and editing of visualization components [41, 42] were also evaluated. As a result, PlanningLines [13] were used as a basic idea to be modified during the design process to regard all needs and a suitable interaction.

Iterative Design Process

The design as well as the interaction ideas were developed on a whiteboard and subsequently implemented as rapid prototypes. In this way, the creative side of developing a new visualisation could be emphasised [39]. The development of the design was an iterative process. Obviously, the design of the Tube itself took most of the time during the process. After the first design approach and a possible input action with a mouse and keyboard were developed, the ideas were implemented within the application. This design showed the advantages and drawbacks of the decisions, which subsequently led to improvements in the interaction design and visual mappings. After the interaction with mouse and keyboard was established, the idea to support also modern devices like tables came up and the interaction to draw Tubes and their components with a stylus were developed. Integrating these ideas into the system leading to a slight modification of the overall design and the application. The iterative process and integration of probability encodings were quite similar to the design process of the Tubes. However, this dependency makes the design more challenging. Thus, a suitable representation of probability distributions within the Tubes was found after a few iterations. The interaction for shadow drawing and editing of Tubes as well as the design of the main application window was continuously developed further along the design process.

At the end of the process, our new approach was evaluated with three different methods: (1) a reading assignment regarding the Tube as a visualization with all facets of uncertainty, (2) a drawing assignment to evaluate the operability of the interactions, and (3) a system usability score to measure the usefulness and benefit of the application.

Visualization Method

As mentioned above, uncertainty is ubiquitous in everyone’s life [3] This is especially the case for the planning of future activities and thus, visualising activities with temporal uncertainty is not a trivial task. For example, a simple holiday trip with five activities features a variety of temporal uncertainties.

  1. 1.

    A flight to the destination has a fixed \(\mathcal {S}\) and \(\mathcal {D}\), and thus fixed \(\mathcal {E}.\)

  2. 2.

    Packing a suitcase before the trip takes at least one day.

  3. 3.

    Sightseeing during the holidays for at least four days but maximum of five days.

  4. 4.

    A two-day trip during the holidays.

  5. 5.

    The return flight is expected to take place on the scheduled day, but could be delayed by one day due to a pilot strike.

Taxonomies [43, 44] and schedules like the holiday trip mentioned above propose the following visualization characteristics that the design of our new approach should fulfil:

  1. C1

    certain activities

  2. C2

    uncertain activities with a certain \(\mathcal {S}\) and uncertain \(\mathcal {E}\) interval and vice versa

  3. C3

    uncertain activities with an open \(\mathcal {S}\) or open \(\mathcal {E}\) (indefinite activities)

  4. C4

    both certain and uncertain activities within a time span

  5. C5

    fixed characteristics for \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\)

  6. C6.1

    a cumulative probability distribution for \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\)

  7. C6.2

    a discrete probability distribution for \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\)

  8. C7

    dependencies between two activities

Fig. 2
figure 2

Representative configurations of Tubes according to the characteristics (Char.) C1–C7. For C5\(^\dagger\), the buffer time has to be used, if the first possible start is taken. The rows marked with \(*\) each show two representatives with different probability distributions. C6.1\(^*\) shows activities, which are probable finished after 25% or 75% of the buffer time. C6.2\(^*\) shows activities for which the whole buffer time happens with a probability of 25% or 75%. In C7, the dependency between two activities is visualized by an orange arrow

Full size image

With PlanningLines [13] the characteristics C1, C2 and C4 are already supported and for the remaining characteristics, they provide a good baseline. Aigner et al. provide simple projects plans with uncertainty in their publication. Nevertheless, their approach does not support interactive input and differs in various visual aspects as detailed below. Therefore, we slightly modify their components to offer the possibility of interactive drawing as well as the editing of activities within a schedule. We named our new development Tube. A simplified representation of a Tube with all its features can be seen in Fig. 3. Examples for different configurations of Tubes fulfilling the Characteristics C1–C7 are shown in Fig. 2.

Fig. 3
figure 3

A Tube with its attributes. The time span of the activity is limited by the start and end interval. The maximum duration is the sum of the minimum duration and the buffer time

Full size image

Since PlanningLines were already successfully evaluated [13], the formal constraints and properties as well as the temporal attributes were adopted to visualize the \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\) properties of an uncertain activity (see Fig. 3). The start interval is bounded by the earliest start time [EST] and the latest start time [LST], the end interval is limited by the earliest end time [EFT] and the latest end time [LFT] and the maximum duration of an activity is defined by the minimum duration plus a possible buffer time—the time difference between the maximum and minimum duration.

The visualization of the buffer time is the main visual difference between the PlanningLines [13] and our new approch. As PlanningLines divides the buffer time equally between both ends we designed our Tubes with the buffer time always visualized on the right side of the minimum duration to foster easier understanding. The user, therefore, does not have to add two parts of the buffer time but can perceive it at once.

As written above, PlanningLines already covers some requirements of the characteristics. To support the remaining demands, several combinations of the visual variables for visualizing uncertainty by MacEachren [3] are used. To come up with a good solution, we worked on several sketches in an iterative process and discussed them during the development of our system (see “Design Process”).

To cover the remaining characteristics, we came up with the following solutions:

  • Indefinite activities (C3) are visualized with an open bracket on the open side. Therefore, the horizontal lines are the remaining parts to foster a more intuitive understanding.

  • To support fixed characteristics for \(\mathcal {S}\),\(\mathcal {D}\) and \(\mathcal {E}\) (C5), \(\mathcal {D}\) is visualized with a texture for a discrete setting. \(\mathcal {S}\) and \(\mathcal {E}\) are visualized as arrows with the direction of the arrowhead indicating whether it is about \(\mathcal {S}\) or \(\mathcal {E}\).

  • For a variable activity with a known probability distribution (C6.1) of \(\mathcal {S}\), \(\mathcal {D}\) or \(\mathcal {E}\), a cumulative distribution function is chosen to visualize the user input using a linear gradient brush (C6.1\(^*\)) (Fig. 9). The calculation is adapted from Correl et al. [31].

  • The probability for a fixed buffer (C6.2) time is represented by a chessboard texture. The granularity is varied based on the set probability, so less probable buffer time is sparser than more probable buffer time (C6.2\(^*\)).

For cases when \(\mathcal {S}\) has a known probability, ambiguation is used, so a less probable fixed start is visualized in lighter colors than a more probable one. This is also applied to a discrete \(\mathcal {E}\). Different Tubes with a constant probability distribution are shown in Fig. 10.

Application

Fig. 4
figure 4

The figure shows a schedule of a holiday trip planned with temporal uncertainty. Our new Tubes visualize the activities with different properties. The user can draw each activity with simple inputs. The corresponding drawing actions are shown in the bubbles near each Tube. Furthermore, the mouse-sensitive context menu for editing is shown at the ”Sightseeing“ Tube

Full size image
Fig. 5
figure 5

Drawing an uncertain activity with the different input methods and the resulting Tube

Full size image

A system architecture was developed to support drawing, editing and visualizing activities with temporal uncertainty. All user controls are grouped into three categories within a menu bar. The main category provides the options for input and editing of Tubes (see “Input of Tubes” and “Editing of Tubes”). As an extension to Aigner et al. [13], we provide the means to draw and visualize both continuous and discrete activities. Furthermore, we also allow the input of probability distribution for all uncertain components of a Tube and offer the possibility to draw indefinite activities. The other menu categories contains I/O features and several possibilities to personalize the schedules (Fig. 4).

Input of Tubes

Two methods for drawing a single activity were developed during the design process: input with mouse and keyboard and input with stylus on a tablet. Both methods offer the same possibilities to draw activities. Therefore, the user can enter either a start interval, an end interval, or a duration with an optional buffer time. If the time span of an activity is known, the user can also enter a time range instead of a start and end interval. The components are drawn as shadows and after finishing the input of all components the user receives suggestions for possible Tubes (Fig. 7). A schematic representation for the input of an uncertain duration with both input methods is given in Fig. 5.

Drawing with Mouse and Keyboard

To draw an activity with mouse and keyboard, the user has to use the control (CTRL) and shift (\(\Uparrow\)) buttons as modifiers. The position of the mouse defines the position of an activity within a schedule. The user can draw the duration with the mouse while pressing the left mouse button and move the mouse in a straight line. The user can also draw Start, End and Range elements in the same way. To draw a buffer time, the user starts to enter a duration and presses the shift button additionally from the point where the buffer time should start and moves the mouse with pressed left button as far as the buffer time should go. During the sketching process, all elements are classified due to their relative position to each other, whereas the first drawn element is always classified as a duration element—regardless of a possible buffer time. The user can monitor its input due to the shadow drawing method [28].

Drawing with a Stylus

As a second method for drawing, the input with a stylus is also implemented to provide a compatibility with modern touchscreen devices, e.g. tablets. In this mode, the user can draw lines with the stylus, which are automatically classified afterwards depending on their shape. To enter a certain activity, the user has to draw a straight line, similar to the movement of the mouse in the alternative mode. Since no keyboard or other additional tools can be used to press certain keys as modifiers, the line classifier has to distinguish between certain and uncertain activities in another way. If there is a point with a slope \(|m| \ge 1\), then the buffer time applies from this point onward (Fig. 5, red circle). Further input elements, Start, End and Range, can be drawn and are analysed by the classifier. The following code outlines the classification, where PC is an ordered collection of 2D points representing the internal structure of the input. The algorithm first extracts three important points of PC: the first point f, the last point l, and the point d with the greatest horizontal distance from f. \(\tau\) specifies a small tolerance value that is given to the user to accommodate when entering a range.

figure a

After classifying the input, the components are displayed as shadows to support the user for further input [28].

Shadow Drawing

We provide a dynamic support to offer instant feedback during the sketching process. This system is adapted from the approach of a guiding system for freeform drawing by Lee et al. [28]. Figure 6 shows an example of our implementation by a Tube with a \(\mathcal {D}\) and buffer time within a known range. Both input with mouse and keyboard and input with stylus are shown. In the first step while drawing \(\mathcal {D}\) the path of the mouse and stylus respectively is displayed to the user to maintain focus. Immediately after the input of the \(\mathcal {D}\) is finished, the input is converted to a shadow of \(\mathcal {D}\) so that the user can draw a range around the duration. After all elements are drawn, the user is supported with the shadow of the input made and a dialog with suggestion for Tube configurations based on it (see Fig. 7).

Fig. 6
figure 6

Input of a Tube with a \(\mathcal {D}\) and buffer time within a known range. The two input methods with their respective shadows during the drawing phase

Full size image

Transforming Input into Tubes

After the drawings are taken—either with mouse and keyboard or with a stylus—and the input is formally correct, the user gets suggestions for possibles Tubes depending on the taken input. Since the input is only clearly defined through the triple of \(\mathcal {S}\), \(\mathcal {D}\) and \(\mathcal {E}\), the user can select the desired result Tube from a pop-up menu that shows all possible combinations for the drawn components. Figure 7 shows an example of the user dialogue for the possible Tubes with drawn user input of a duration (with buffer time) and a range.

Fig. 7
figure 7

An example dialog for the supervised input. The user has to choose the Tube he wants to draw with the input of an uncertain duration and a range

Full size image

Editing of Tubes

Editing drawn Tubes is an essential feature of the interactive input of activities, especially for Tubes with characteristic C5 or C6. Beside the trivial actions like deleting or moving a Tube, more complex operations for editing Tubes and creating dependencies between two Tubes are implemented. To open a context menu for editing a Tube, the components are supplied with mouse-sensitive editing points in the corresponding editing mode of the software. Figure 8 shows this behaviour for the buffer time of an activity. You can either or remove a probability distribution (left), add a probability distribution (center) or change this buffer time from variable to fixed or the other way round (right).

Fig. 8
figure 8

A context menu for editing of this Tube to remove a probability distribution (left), add a new probability distribution (center), or change the buffer time from a continuous to a discrete one (right)

Full size image
Fig. 9
figure 9

The user can set a known probability, whether an activity is more likely to last longer or shorter, with a slider in the probability distribution function (top left) and sees the resulting cumulative probability function directly (top right). Bottom shows the resulting Tube with the set probability distribution in the buffer time visualized with a gradient

Full size image

Probability Settings

The user may want to quantify the probability distribution of \(\mathcal {S}\), \(\mathcal {E}\) or the buffer time in some cases. After selecting the corresponding menu item (see Fig. 8, center) a new user dialogue pops up (Fig. 9) in which the user can set, whether the activity is more likely to last longer or shorter. The expected duration can be set with a slider in a probability distribution function (left, black). On the right side, the result of the input is shown directly as a cumulative distribution function (blue). The cumulative distribution is converted into a gradient [31] to show the probability distribution in the resulting Tube (bottom). The same technique can be used to quantify a probability distribution for the \(\mathcal {S}\) and \(\mathcal {E}\) components of a Tube (see Fig. 2).

Fig. 10
figure 10

Tubes with a constant probability distribution from 1.0 to 0.0 (top row, left to right) and Tubes with a discrete buffer time and a respective probability from 1.0 to 0.0 (bottom row, left to right)

Full size image

Variable and Fixed Components

To quantify a constant probability for the buffer time (i.e. ‘every point in time has the same probability’) or a probability for fixed components in an activity (\(\mathcal {S}\), \(\mathcal {D}\), or \(\mathcal {E}\)), the dialogue only has to offer one slider to set the desired probability. The result for a constant probability with a fixed buffer time and different probabilities is shown in Fig. 10. To visualize a quantitative probability for a fixed buffer time, we chose the technique grain by MacEachren [3], where the rule is: the less likely the buffer time, the sparser the grid. For Tubes with a variable buffer time, the probability is mapped to the alpha value of the base color. The target function has a value range from 0.1 to 1.0 to avoid totally transparent components (zero probability). The same color is chosen for the base color and the color for the minimum time, so a buffer time with 100% probability is visualized like the minimum duration. Representatives for different probabilities are shown in Fig. 10. For fixed \(\mathcal {S}\) and \(\mathcal {E}\) components, the same technique as for variable buffer time with a constant probability is used. The components differ in their alpha value, depending on the chosen probability (see Fig. 11).

Fig. 11
figure 11

Tube with discrete start and end and constant probability. The activity happens with 70% in time span (1) and 30% in time span (2)

Full size image

Additional Features

Fig. 12
figure 12

In the red-marked time spans, the Tubes conflict with each other, because Tube 1 and Tube 2, respectively Tube 1 and Tube 3 definitely happen there in this schedule

Full size image

To avoid conflicts within a schedule, the application has a built-in cross-check to highlight such risks. The system examines the drawn Tubes pairwise after each input to detect overlaps within the schedule. For this, only the part of an activity that will certainly happen is taken into consideration. If a conflict is detected, the corresponding time span will be highlighted with a red marker (see Fig. 12). Furthermore, Tubes can be moved on the x-axis to re-define their time span within the schedule. They also can be moved on the y-axis into a different layer to avoid overplotting. As with PlanningLines [13], the user also has the opportunity to draw dependencies between two activities (see Fig. 2, C7) to visualize that one activity has to be finished before the other one can start.

Evaluation

To evaluate our implemented approach we conducted a three-part quantitative user study. The main objectives were to evaluate the drawing input of the new application and the corresponding visualization. Therefore, the participants were asked to perform two different drawing assignments to explore the potential of the input methods and the application. The second part of the evaluation dealt with the visualization itself. The participants were asked to answer question on specifics of activities within a given schedule. In the end, the participants were asked to fill in a system usability score to measure the benefit of the new application.

Experimental Setting

A total of 21 participants (10 male, 10 female, 1 preferred not to say) took part in the study, the age ranging from 20 and 71, with the majority of the participants between 20 and 30 years old (67%). The participants’ professions covered a broad spectrum of users, ranging from architects to computer scientists to teachers. We conducted the study as a laboratory study. The study conductor was present at all times to answer any technical questions. All participants used the same technical equipment to ensure equal conditions. Each participant worked on all assignments, following a within-subject study design. This approach generates a significantly larger result set than a between-subject design with the same number of participants [45]. The evaluation started with a brief introduction to the subject of the study and an explanation of the user interface by the conductor. Users were asked to use the built-in help system to answer their questions concerning the application and the visualization by themselves. All participants took between 60 and 150 min to complete the study.

Initially, we split the participants into two groups—the first group starting with the reading assignment, followed by the drawing assignment and the second group starting with the drawing assignment and finishing the reading assignment. The underlying hypothesis of splitting the participants this way was that participants would make fewer mistakes when drawing if they had seen and understood the visualization beforehand by finishing the reading assignment first. However, our initial evaluation showed that both groups completed the task with almost identical correctness, suggesting that there is no significant difference in the learning effect. Taking these findings into account, we considered all participants as one group for the following result evaluation.

Assignments and Data

Fig. 13
figure 13

Tubes with different characteristics representing a hypothetical day at a university

Full size image

The drawing assignment addresses the intuitiveness and robustness of the newly developed software. Besides the known use case (Fig. 4), a second schedule had to be visualized (Fig. 13). For this schedule, a hypothetical day at a university was created with the following activities:

  • Visit a lecture between 10:00 am and 11:00 am.

  • Exam preparation with fellow students at 10:30 am with an uncertain end between 12:30 pm and 01:30 pm.

  • Visit an exercise starting at 02:00pm with a duration of at least 45 min. After 45 min, the assignments are probably not yet completed. However, the chances increase with each minute. In any case, the exercise will take a maximum of 90min.

  • Go to the canteen for a maximum of 45min after the lecture, but before the exercise starts.

  • Take the bus after the exercise. It will take either 15min or 30min, depending on the route.

These two schedules (“a day at a university” and “a holiday trip”) were given to the participants in a textual representation and the participants were asked to draw them using the new application. Each schedule consisted of five activities focusing on different specifics, e.g. certain start and uncertain end, or activities containing a probability distribution or discrete components.

Fig. 14
figure 14

The example schedule of the reading assignment. The plan has been adopted from Aigner et al. [13] and slightly modified so that all characteristics can be queried

Full size image

The participants were also given a reading assignment as a prepared schedule with different activities, drawn in advance by the conductor with the new application. To allow a comparison with the results of PlanningLines evaluation, the schedule presented in the assignment is a slightly modified version of the project plan evaluated by Aigner et al. in 2005 [13]. We have varied the activities so that all characteristics are represented by at least one Tube. The schedule is depicted in Fig. 14.

This task was carried out to evaluate the understandability and clarity of the visualization. The participants were asked to answer questions on the details of the visualizations, e.g. “When is the earliest possible start?”, “When is the latest possible end?” or “What is the maximum duration of the activity?”.

Finally, the participants were asked to fill in a form for the system usability score by Sauro [46].

Analysis

Fig. 15
figure 15

Results of the drawing assignments (D1, D2) and the reading assignment (R). The assignments had a similar average rating, but differ strongly in the variance of the results

Full size image

The drawing assignments were evaluated by the specific features of the Tubes to be drawn. An existing property was assigned a value of 1 and a missing property was assigned a value of 0. The reading assignment was evaluated with 1 for a correct answer and 0 for a wrong answer. Hence, an average correctness value of 1.0 means that all participants completely fulfilled all requirements, while a value of 0.0 means that no requirement was fulfilled by any participant. The results are shown in Fig. 15 and are explained in more detail throughout the remainder of this section.

Drawing Assignment

The two schedules (D1—a day at a university, D2—the holiday use case) were evaluated individually, followed by a comparison between them.

D1 has an average correctness of 0.8413. The only activity with a comparably low correctness (0.7048) was the bus transfer. The main problem was to set the right start interval. The intended solution was an uncertain start within 45min and 90min of the duration of the exercise ahead. Instead, eleven participants drew a certain start right after the latest possible ending of the exercise. Moreover, the probability distribution for this fixed and uncertain activity was not set correctly in 12 of 21 cases. To enter the probability distribution correctly was also a major issue for the exercise activity—10 of 21 participants had problems with that aspect.

D2 has an average correctness of 0.7755. The main problem with this schedule came from the indefinite activity of packing the suitcase (0.7738) and the inbound flight (0.6667). In the suitcase activity, setting the right start was the main challenge, whereas the inbound flight causes multiple problems concerning the maximal duration, the fixed start and end interval, and the probability distribution.

Both drawing assignments have exactly the same median correctness value of 0.8095. This high number demonstrates the ease of use of the developed software, especially for new users. Participants did not have much trouble entering certain activities, nor did they have much difficulty entering uncertain activities within a time span. Problems occurred only with activities with more specific properties, such as fixed components or probability distributions.

Reading Assignment

The reading assignments have an average correctness of 0.7875. The results indicate that users read the certain activity almost without error, just like the Tube with fixed components. Problems occurred with the indefinite activity and its open end. The major problems occurred with the probability distributions and with the components \(\mathcal {S}\) and \(\mathcal {E}\), which caused participants significantly more problems than such a characteristic with \(\mathcal {D}\).

Summary

The two different tasks show similar performance for the different properties of the tubes. On the one hand, participants had difficulty with the probability distributions, both in drawing and reading. This could be due to the fact that judgements under uncertainty are often conveyed by using intuitive heuristics [47]. In addition, 10 participants drew constraints between two activities. Although this was not a task in the assignments, it is a good point to address in future iterations of this research. On the other hand, Tubes with characteristics apart from probability distributions did not seem to be a problem for the participants, both in drawing and in reading. This observation is also reflected in similar statistics for the tasks (see Fig. 15). With an average correctness of 0.8117, the study shows good results both for drawing plans with our software and for visualization. Therefore, only 18.83% of the given answers deviated from the desired input.

Fig. 16
figure 16

The distribution of the System Usability Score for the evaluation of the new application annotated with the adjective ratings according to [48]. There are a few outliers \(\circ\), but the majority of the participants rated the system as ‘ok’ or ‘good’, reflecting in the mean \(\diamond\) and median value

Full size image

System Usability Score

The system usability score indicates the subjective evaluation of the usefulness and utility of the application by the participants [46]. The score of this evaluation for our system is between 30.0 and 92.5. This range is shown in Fig. 16 together with the median value and the classification by Bangor et al. [48]. The average score of 66.19 shows that our new application can be classified as ‘OK’.

A detailed overview of the ratings of all participants for all questions can be seen in Table 1.

Table 1 Results of the System Usability Score [46] with the 21 participants (P)
Full size table

Conclusion and Future Work

We presented a new visualisation for planning with temporal uncertainty and a corresponding application for an interactive visual input of such plans. As a basic visualization, we used the PlanningLines approach by Aigner et al. [13]. We have extended and modified this approach so that it is possible to visualise both variable and fixed activities. It is also possible to visualise various probability distributions and indefinite activities. While sketching a Tube, the user is assisted with shadow-drawing [28] to provide a visual aid for completing the desired Tubes. We conducted a quantitative user study to demonstrate the added benefit of our new visualisation and application. The drawing assignments underline the advantages of the new application in externalising the temporal uncertainties. The reading assignment demonstrated the appropriateness of the visualisation. Moreover, the user study revealed that the holiday use case example (see “Introduction”) can be externalized by the majority of the participants without significant difficulties. The objective evaluation shows an average correctness of about 80% for both assignments in the study. Since no participant used the application before the study took place, it can be assumed that the accuracy will increase with regular use. To assess subjective perceptions, participants were asked to complete a system usability score [46]. The score shows, that the system can be classified as ‘OK’. However, the wide range of ratings shows that we need to improve the user-friendliness of entering complex configurations.

In the next steps, additional functions for editing Tubes will be developed and implemented. This includes possibilities to vary the duration of the components of a Tube as well as an advanced input dialogue to enter the probability distributions in an easier way. We are also planning advanced quality-of-live enhancements such as zooming and panning, or individual colors for individual Tubes to offer more customisation options for the schedule.